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Executive Summary 

In 1996, the Chicago Public Schools 
(CPS) began implementing a new school 
accountability policy designed to 
improve student performance by 
providing a combination of 
consequences and support to low- 
performing schools. The center point of 
the accountability system, the Chicago 
school probation policy, designates 
schools as being "on probation" if fewer 
than 15% (later raised to 20%) of their 
students score at grade-level norms on 
the Iowa Test of Basic Skills in reading. 
When placed on probation, schools face 
the consequences of decreased 
autonomy and the threat of more severe 
sanctions. At the same time, probation 
schools receive direct assistance from 
several different sources through the 
policy's external support system. The 
purpose of the support is to assist 
schools in strengthening their internal 
operations, raising expectations for 
students, and improving instruction so 
as to foster increased student 
achievement. 

This report is based on a two-year 
study of the design and implementation 
of the school probation policy in 
Chicago's elementary schools. The 
school accountability system in Chicago 
has undergone changes since the end of 
this study as a result of new district 
leadership. The system now includes the 
use of additional assessment data and 
subject area tests, emphasis on progress 
and growth, and a focus on all schools. 
In spite of these changes, the assistance 
provided by probation managers and 
external partners has not changed. 
Therefore, the lessons learned from this 



study should still be relevant not only to 
CPS but also to other jurisdictions 
instituting similar policies. 

Overview of External Support to 
Probation Schools 

The Chicago school probation 
policy's external support system 
includes five school-level support 
providers: an external partner, a 
probation manager, a regional 
education officer, a business manager or 
intern, and a facilitator from CPS's 
Office of Accountability. For this study, 
we focused on the role of the two 
external agents — the external partners 
and probation managers — because 
they are the central components of the 
system. Schools select their external 
partners, which tend to be either 
universities or individual consultants, 
but the seasoned administrators known 
as probation managers are assigned to 
one or two schools by the district. 
Although neither role is specified in 
great detail, external partners tend to 
provide professional development and 
work with staff on aspects of school 
improvement, while probation 
managers mentor and supervise the 
principal and oversee the 
implementation of the school's 
improvement plaiming process. 

Several characteristics of Chicago's 
design of external assistance are 
noteworthy. Support is a -prominent 
feature of the probation policy — 
recognizing the need for capacity 
building in low-performing schools, 

CPS spent approximately $29 million on 
this effort between 1997 and 2001. The 
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policy emphasizes the provision of site- 
based assistance in order to improve the 
school as an educational unit. Instead of 
generic professional development 
activities, teachers receive professional 
development as part of a school-wide 
improvement plan, and most of the 
assistance occurs on site. The policy 
includes multiple avenues of support and 
the separation of the probation manager 
and external partner roles. Most of the 
policy's external assistance — that of the 
external partners — is market based, 
allowing for a better "fit" between their 
needs and the external partner's 
program because they select and 
contract with these independent 
providers. A final aspect of the policy is 
a relative lack of emphasis on training of 
support providers and the absence of 
built-in mechanisms for learning, either 
on the part of the providers or the 
district. 

Provider Strategies and 
impiementation of Assistance 

External partners and probation 
managers locate the problem of low 
performance within the schools — either 
at the classroom or organizational level. 
However, beyond pointing out the 
internal nature of the problem, support 
providers vary in their description of its 
source or solution. When targeting 
individual (classroom) capacity, external 
partners primarily focus on teachers, but 
most partners also provide some type of 
materials, from assessments to lesson 
plans. While most partners target the 
development of teachers' knowledge 
and skills, few emphasize content 
knowledge and learning. Most 
professional development takes the 



form of traditional half-day or full-day 
workshops with more limited attention 
to one-on-one assistance and modeling 
in classrooms. To the extent that 
external partners focus on the 
environment or organization to build 
capacity, their interventions frequently 
overlap with those of probation 
managers. The organizational-level 
efforts of both groups targeted one or 
more of four areas: coordination and 
leadership, professional norms for 
teachers and teaching, the monitoring of 
instruction and student learning, and 
resources for learning and instruction. 

Implementation Constraints 

Our evidence clearly points to the 
fact that all probation schools receive 
assistance from a probation manager 
and external partner. The variation we 
observe in these providers' strategies is 
consistent with the market-based 
approach, and most respondents report 
general satisfaction with the support 
system. 

However, our analysis suggests 
several problem areas that we believe 
limit the impact of this support. 

. Low levels of intensity. While 
varying among providers, the 
intensity of assistance delivered to 
probation schools is, we believe, 
simply inadequate to address the 
substantial needs of the individuals 
and organizations concerned. Some 
partners attempted to extend the 
reach beyond the time that the 
consultants were in the schools 
through train-the-trainer approaches 
and on-site literacy coordinator 
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models. Our observations in case 
study schools suggest that 
professional development in the 
train-the-trainer model was much 
too weak to extend into other 
teachers' practices, while the on-site 
literacy coordinators hold more 
promise. 

• Lack of communication among 
providers. According to our 
respondents, the lines of 
communication generally flowed 
from central office persormel to 
providers with little opportunity for 
feedback or for open communication 
and collective problem-solving 
among external partners, among 
probation managers, or between 
partners and probation managers. 
We saw similar communication 
problems mirrored in our case study 
schools. 

. Uneven targeting of literacy 
instruction. One rather surprising 
finding was that most support 
providers implement strategies that 
only weakly attempt to improve 
literacy. Only 4 of the 11 partners 
included in this study had 
developed a comprehensive, 
research-based approach to literacy 
instruction. 

. Lack of clear strategies for 

organizational change. While the 
goal of the assistance was to develop 
capacity and alter practices 
throughout the school unit, support 
providers generally lacked a 
comprehensive intervention or 
dissemination strategy based on an 
articulated theory of action or 



understanding of organizational 
change. 

. Reliance on traditional staff 

development models. Most partners 
continued to rely on traditional 
forms of staff development — 
primarily short-term workshops — 
with little intensive assistance to 
teachers, little examination of 
student work, and few ongoing 
opportunities for cross-grade, cross- 
class, and cross-school collaborative 
learning. 

Strength of External Support 
and Implications 

Through its allocation of substantial 
resources and attention to the external 
assistance efforts, CPS has demonstrated 
not only a recognition of the need for 
capacity building in its low-performing 
schools but also a commitment to its 
realization. Few jurisdictions have, from 
the outset, designed an accountability 
system with this level of commitment to 
school assistance. Moreover, the 
incorporation of multiple layers of 
assistance reflects the district's 
recognition of the variability in school 
contexts, focus on the "fit" between the 
providers and the schools, and attempt 
to involve the broad education 
community, including institutions of 
higher education, in improvement 
efforts. 

Unfortunately, despite the millions 
of dollars put into assistance, we 
conclude that the support component of 
the probation policy is simply too weak 
to make it through the many layers of 
implementation to significantly alter 
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classroom instruction. We find the 
assistance system to be under-specified 
with respect to the target and goal of the 
assistance, to lack clear authority or 
incentives to encourage participation, to 
suffer from inconsistencies and even 
conflicts with other policies, and to be of 
generally too low intensity to induce the 
kinds of learning and change needed at 
the level of classroom instruction. 

Based on our analysis, we suggest 
seven implications for policy: 

1. Constrain the market choice, both 
with respect to direction and in 
terms of quality. We believe the 
provision of assistance would be 
improved if the number of partners 
were limited to a handful of proven 
groups that were required to offer or 
support a coherent and 
comprehensive instructional 
program at least in the area of 
literacy. In this more constrained 
market approach, the district could 
better ensure both quality and 
appropriate variation and flexibility, 
assisting low-capacity schools to 
make more useful choices. 

2. Clarify roles of support providers. 

The existence of multiple partners 
and providers almost guarantees 
fragmentation at the school level. We 
suggest that the roles of probation 
managers, regional education office 
staff, and Office of Accountability 
facilitators be consolidated into a 
single and clear line of authority in 
the schools and that this authority 
work closely with the external 
partner to help schools develop and 



implement a coherent instructional 
program. 

3. Develop opportunities for 
learning/sharing among partners 
and probation managers. We 

strongly urge the district to design 
professional development for these 
support providers and opportunities 
for them to reflect collectively on 
their work, its effects, and resulting 
lessons. 

4. Discourage schools from 
developing multiple and 
fragmented partnerships by giving 
the external partner the authority to 
coordinate assistance from all 
external sources while the school is 
on probation. 

5. Stimulate the development of in- 
depth, content-based professional 
development. An example of such 
staff development might be the two- 
week session on reading instruction 
for Targeted Assistance Program 
teachers held in the fall of 2000. 
Despite the implementation 
problems of short notice and 
inconsistency with some literacy 
programs, this effort to foster deeper 
content and pedagogical content 
knowledge in literacy was an 
important step and should be 
pursued further. 

6. Connect assistance to the standards, 
not just the test. We observed a real 
problem of teaching to the test in 
several of our case study schools, a 
problem that is exacerbated by the 
lack of attention to the standards and 
the reliance on a single indicator for 
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both student promotion and school 
probation. As long as the Iowa Test 
of Basic Skills is the sole indicator 
and everything rides on it, it will 
become the de facto standards of the 
district. 

7. Promote better use of data in schools 
by fostering the development and 
implementation of multiple 
diagnostic tools such as ruiming 
records, periodic curriculum-based 
assessments, and examination of 
more extensive examples of student 
work. 
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Introduction 

In 1996, the Chicago Public Schools 
(CPS) made history by placing 109 of its 
lowest performing schools on academic 
probation through its accountability 
policy known as the Chicago school 
probation policy. The goal, simply, was 
to shake up the system and improve 
student performance. Of the schools put 
on notice, 71 were elementary schools^ 
and 38 were high schools. Although the 
magnitude of the CPS action was unique 
(a full one-fifth of its schools were 
placed on probation), the underlying 
CPS policy mirrors educational 
accountability systems instituted in 
many states and urban districts in recent 
years. 

A hallmark of Chicago's policy is the 
combination of consequences and 
support. This combination is meant to 
improve student performance by 
increasing both the will and the capacity 
of educators to teach well. The 
consequences of probation include a 
decrease in autonomy while the school 
is on probation and the threat of more 
severe sanctions (reconstitution or re- 
engineering) should student 
achievement not improve in a 
reasonable period of time.^ The support 



' Most elementary schools in Chicago include 
kindergarten through eighth grade. 

^ Reconstitution refers to the removal of staff. 
Re-engineering, CPS's subsequent incarnation of 
reconstitution, consists of a year-long 
assessment period during which the district 
decides whether to hire and fire staff. Each 
school is given $500,000 in resources during the 
assessment period. No elementary schools had 
been targeted for reconstitution or re- 
engineering at the time of this study. 



component employs multiple groups 
and individuals within and outside the 
district to provide assistance to 
probation schools. 

This report focuses on the provision 
of external assistance to probation 
schools. Our examination of this 
component is based on a two-year, 
multi-level study of the design and 
implementation of the school probation 
policy in Chicago's elementary schools. 
Following this introduction, we provide 
an overview of the role and system-level 
design of external assistance. We then 
outline the analytic frames and data 
sources that informed this investigation. 
Next, we describe the design and 
implementation of assistance at the 
support provider level. We conclude 
with a discussion of the overall strength 
of the support system and implications 
for policy and practice. 
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Study Methods and Data 

This report is based on data from multiple sources from July 1999 fhrough February 2001. 
The study team began by interviewing administrators in CPS's Office of Accountability to 
better understand the origin and evolution of the school probation policy. This report is 
primarily based on the interviews and observations of 11 of fhe 18 external partners 
providing support to schools during this time period. We interviewed both external partner 
directors and consultants and shadowed a subset of five partners serving the bulk of 
probation schools. The partners participating in this study are listed in Appendix A along 
with the number of schools associated with each at the time. 

This report is also based on interviews with a random sample of 16 (of the 53) probation 
managers who were working with at least one probation school at the time of data 
collection. These interviews focused on the ways in which probation managers monitored 
school performance and assisted principals with school management and leadership. Other 
data sources for this report include interviews with four Office of Accountability facilitators 
who worked with approximately 30 probation schools, reviews of district documents 
relating to school probation and the system of external support, and observations of 
support provider meetings and workshops when possible. 

To better understand the school-level perspective, we interviewed a random sample of 15 
principals at probation or post-probation elementary schools about their experiences with 
external partners and probation managers and their satisfaction with the support provided 
through this policy. In addition, this report draws on interviews and observations at the 10 
schools involved in our case study analysis. It is important to note that describing the 
system of external support is a difficult task because it has been a moving target. On the one 
hand, the district has changed its policies over time. On the other hand, turnover in district 
staff, external partners, and probation managers proved challenging to this study. 
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Overview of External 
Support to Probation 
Schools 

Support as an Integral Aspect 
of the Probation Policy 

Although this report focuses almost 
exclusively on the policy's system of 
external support for elementary schools, 
it is important to understand how this 
component fits within the overall policy 
design. This section provides a brief 
overview of the design, assumptions, 
and theory of action of school probation 
in Chicago. 

Low performance has plagued 
Chicago schools for many years and has 
inspired multiple critiques and policy 
responses. The most recent of these 
responses — direct school accountability 
for student performance — is a result of 
the convergence of both local and 
national factors. Locally, a shift in the 
city and state political contexts in the 
early 1990s gave the mayor and his 
newly appointed Chief Executive 
Officer (CEO) greater authority to 
intervene in troubled schools.^ At the 
same time, the national political 
landscape was also shifting toward 
higher standards, more rigorous 
assessments, and increased 
accountability for student learning. 
Consistent with this larger political 
landscape, CPS began tightening control 
over its schools, defining accountability 
in terms of student test scores, and 
placing schools with the lowest student 



^ See Bennett (2001) for a detailed account of fhe 
political confexf of fhis policy. 



performance on probation until that 
performance improved.^ While the 
probation policy appeared negative 
(largely because of the term "probation" 
and the stigma associated with it), the 
district described it as a "positive step," 
emphasizing a shift to the needs of 
students and the provision of support to 
low-performing schools (Chicago Public 
Schools, 2000b). 

Probation Policy Theory of Action 

District administrators interviewed 
for this study attributed low 
performance in probation schools to 
several factors: poor principal 
leadership, inadequate instructional 
knowledge and low expectations on the 
part of teachers, and lack of motivation 
for improvement.® School probation 
represents a central district response to 
these problems. As such, the school 
probation policy rests on a set of 
underlying assumptions common to 
recent accountability approaches. These 
assumptions taken together make up a 
general "theory of action" (Argyris & 
Schon, 1978), the broad components of 
which are as follows: 

. The accountability system should 

be aligned with educational goals. 

If increased student achievement is 

the goal, then student achievement is 

what educators should be account- 



“ Authority for the probation policy derives from 
a 1995 revision of Illinois legislation on Chicago 
school reform. 

^ Preliminary interviews were conducted in 
spring 1997, shortly after the first schools went 
on probation. Subsequent interviews of district 
personnel occurred from March 1998 through 
spring 2001. 
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able for and what the system should 
monitor. The belief is that the 
accountability system will draw 
attention to the desired goals and 
focus effort on their achievement. In 
Chicago, these goals are measured 
primarily through the Iowa Test of 
Basic Skills (ITBS). 

• The school should be the basic unit 
of accountability. Two beliefs 
support the school as the unit of 
accountability. One is that student 
performance is cumulative and 
influenced by the entire school 
program and climate. A second is 
that a system of collective (school- 
based) accountability will encourage 
educators at the school to work 
together, thus increasing overall 
capacity and performance. 

. The threat of negative sanctions 
will motivate educators to work 
harder, thus improving student 
achievement. The underlying belief 
here is that persistently low student 
performance stems primarily from 
educators' lack of will. Negative 
sanctions are to provide the extra 
push for school persoimel to do what 
is necessary to raise student 
achievement. 

. Accountability measures should be 
accompanied by capacity building. 

CPS administrators recognize that 
limited capacity (knowledge, skills, 
and resources) at the school also 
contributes to low performance. By 
providing assistance from outside 
the school, CPS administrators hope 
to create the capacity needed to 



improve instruction and student 
learning. 

. Goal setting, planning, and 
monitoring results are critical to 
improvement. The underlying belief 
is that very broadly defined goals do 
little to focus attention, suggest 
strategies, or provide feedback for 
improvement. Schools need specific, 
measurable goals and well-specified 
plans for achieving them. In 
Chicago, this process is incorporated 
into the aimual School Improvement 
Plan for Advancing Academic 
Achievement, required of all schools, 
and the reading plan, required of 
probation schools. 

As illustrated in Figure 1, the 
implicit theory of action is that threat (or 
consequences), combined with support, 
will increase educator motivation and 
capacity to improve performance.^ 
Improved educator performance will, in 
turn, lead to increased student 
achievement, the ultimate goal of the 
policy. As is evident, assistance to 
schools is an integral part of the theory 
of action behind probation — it is this 
assistance that will enable the 
improvement policymakers seek. 



^ Unlike many other jurisdictions at the time, 
CPS also recognized in the mid-1990s the 
importance of attending to student motivation 
and capacity and chose to address these issues 
through the student promotion and retention 
policy and through provision of additional 
learning opporfunifies in summer and affer- 
school programs. See Roderick, Bryk, Jacob, and 
Easton (1999) for research on grade promofion 
and refenfion practices and resulfs. 
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Figure 1. Simplified Theory of Action Underlying CPS Probation Policy 




The Design of External Support 

The district's external support 
system, described by CPS as "targeted 
support," was designed with one 
objective in mind: to improve student 
achievement as measured by 
standardized tests (Chicago Public 
Schools, 1999). The system includes five 
school-level support providers: an 
external partner, a probation manager, a 



regional education officer, a business 
manager or intern, and a facilitator from 
CPS's Office of Accountability. Figure 2 
illustrates the multi-faceted system of 
external support. For this study, we 
focused on the role of the two external 
agents — the external partners and 
probation managers — because they are 
the central components of the support 
system. 



Figure 2. The Design of External Support 
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External Partners 

The most intensive provision of 
support to probation schools is through 
the external partners. According to 
district administrators, the strategy for 
the external partners was intended to 
combine district resources with a 
commitment from external agencies — 
particularly institutions of higher 
education — through aimual contracts 
with each school (see sidebar below for 
other models of external assistance to 
low-performing schools). To be 
approved as an external partner, a 
college, university, or educational 
organization must submit a three-page 
proposal stating its "program" or 
approach. Although the district does not 
require that every external partner 



address each area, the RFP (Request for 

Proposals) emphasizes the following 

goals for the partners' work: 

1. Increase student achievement, 
particularly in reading and math as 
measured on standardized tests; 

2. Improve school leadership, 
including school organization and 
fiscal management; 

3. Establish a student-centered learning 
environment; 

4. Provide effective professional 
development opportunities; and 

5. Promote parent/ community 
partnerships. 



Models of External Assistance 

Chicago's model of devolving authority to the school site to choose an (approved) 
assistance provider that is external to both the school and the system is but one approach to 
supporting capacity building in low-performing schools. Other jurisdictions have chosen to 
bring expertise to low-capacity schools through differing combinations of external and 
internal assistance. In Kentucky, for example, the state education agency established a core 
of "distinguished educators," successful teachers and administrators, throughout the state 
who were selected, trained, and paid by the state education agency to work for two years 
with the principal and staffs of "schools in decline" (Kentucky's equivalent of probation). 
These distinguished educators were individuals external to the low-performing school but 
internal to the state education system. In Baltimore, by contrast, district leaders have 
contracted with Achievement First — a program of the local education fund — to provide 
comprehensive, literacy-based professional development and school intervention to low- 
performing (reconstitution-eligible) schools in the CEO district. In this case, the agent is 
external to the system but contracted by the district (rather than the schools themselves) to 
provide a specific program of intervention and capacity building to a group of targeted 
schools. San Diego employs yet another model with district-trained literacy coordinators as 
full-time staff in schools and peer coaches /staff developers to work with literacy 
coordinators and principals. In this case, assistance is internal to the school and the larger 
system, and it is multi-layered to incorporate learning among coordinators and other 
assistance providers. 
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"After completing and submitting a 
proposal, each partner is selected based 
on its ability to raise student 
performance and its ability to customize 
the assistance to meet the individual 
needs of each school" (Chicago Public 
Schools, 1999)7 Once approved, the 
partner's name appears on a list of 
candidates from which probation 
schools may choose. At any one time, 
approximately 16-18 groups have been 
authorized as external partners. 
Although the central office assigned 
partners to some schools during the first 
year of probation, by the time of this 
study, the choice had been entirely 
delegated to the schools themselves. 

One rationale for this flexible, market- 
based approach to assistance was 
reportedly to allow for a better "fit" 
between the needs and philosophy of 
the school and the approach of the 
service provider. 

Since the inception of the policy, the 
district and the schools have shared 
funding for the partner's services in a 
graduated approach. The first year of a 
school's partnership with an external 
partner is fully funded by the Office of 
Accountability. In the second year, the 
school must assume one-half the cost of 
the partnership, and in the third year 
(and thereafter) the school is responsible 
for the full amount. In addition, after a 
school is removed from probation 
(regardless of the point in this three- 
year process), it must continue its 
relationship with a partner for one full 



' The criteria used to determine whether the 
prospective partner can accomplish those tasks, 
however, remains unspecified in any district 
publication. 



year so that the supports to the school 
are not withdrawn immediately.** 
External partners must provide CPS 
with invoices, logs, and other materials 
to document their assistance. 

Probation Managers 

The Office of Accountability also 
assigns a probation manager to each 
school on probation. According to Phil 
Hansen, former director of the Office of 
Accountability, "they are our eyes and 
ears" at the school site.^ Probation 
managers must be seasoned 
administrators — a current principal, a 
retired principal, or a district 
administrator — with proven success as 
a building leader. The assumption here 
is that success in their own schools or 
districts has provided these 
administrators with the expert 
knowledge needed to guide or advise 
the principal of the low-performing 
school. Half of the probation managers 
who participated in our study were 
former principals who had also served 
in other positions at the district and 
university level (e.g., central office staff, 
university faculty, or superintendents of 
neighboring districts). Most of the 
others were current principals. As of 
May 2000, the responsibilities of the 
probation manager (listed in district 
documents) included monitoring the 
School Improvement Plan for 
Advancing Academic Achievement and 



“ Since 1999, many probation schools have been 
able to gamer additional federal resources to 
support their work with external partners 
through the Comprehensive School Reform 
Demonstration program. 



" Interview, March 27, 1997. 
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the school's financial situation, leading 
the probation management team,“ and 
mentoring the principal. 

Before probation managers begin 
working with their schools, the Office of 
Accountability provides them with 
general information about probation, as 
well as the reporting requirements of 
this position. No formal training for the 
position has been required or provided, 
however. At the time of this study, each 
probation manager received a stipend of 
$5,000 for every school they assisted. In 
recent years, CPS limited the maximum 
number of schools for probation 
managers. For example, probation 
managers in our study commonly 
worked with one school each, although 
a few assisted two. In the early years, a 
few probation managers assisted as 
many as four or five schools at a time. 

As shown in Figure 2, three other 
individuals work with probation 
schools as part of the support system. 
The school business manager or intern is 
responsible for business activities (such 
as ordering instructional materials and 
assisting with the budget), thus freeing 
the principal to focus on instructional 
tasks. A person from the regional 
education office serves as a liaison 
between the school and the Office of 
Accountability, while one or more 
Office of Accountability administrators 
assess the school through site visits and 



“ The manager leads a team at each probation 
school that includes the partner, current 
principal, local school council representative, 
school business manager or intern (at discretion 
of the principal), and regional education office 
represenfafive. This feam is required fo meef 
monfhly. 



operate as "facilitators" between the 
support providers and the schools. 

The focus of this study is the role of 
the primary assistance providers, the 
external partners and probation 
managers. These groups work with 
individual schools but, by design, have 
differing roles and targets. External 
partners are expected to provide 
professional development and work 
with staff on aspects of school 
improvement, including the school 
plaiming process. Probation managers 
mentor and supervise the principal and 
oversee the implementation of the 
School Improvement Plan for 
Advancing Academic Achievement. In 
the first few years of the policy, CPS 
permitted these two groups to function 
under one roof; that is, external partners 
could also be probation managers. CPS 
no longer allows partners and managers 
to serve a dual role because it reportedly 
received complaints about the services 
provided. In doing so, the district also 
began asking probation managers to 
monitor the external partners 
informally. As illustrated in Table 1, 
further delineation has occurred over 
time, although there continues to be 
overlapping responsibilities in many 
probation schools. 

Reliance on external assistance for 
probation schools developed in 
response to two considerations: the 
limited capacity of the central office to 
provide support to nearly one-fifth of 
CPS's schools and the district's interest 
in engaging outside groups, especially 
universities, in school improvement in 
Chicago. The district sought to 
strengthen relationships with 
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Table 1. Probation Support System 

Support Provider Role 

External Support Partner • Educational organization that assists schoois in the improvement of 

student achievement. 

• Works with schooi staff on appropriate issues. 

• Assists in ensuring good schooi/community reiations. 

Probation Manager • High-level school administrator with principalship experience. 

• Oversees implementation of modified school improvement plan. 

• Monitors school educational progress. 

School Operations • Experienced business executive for high schools; interns for most 

Manager/School elementary schools. 

Business Manager Intern , Manages operational and financial aspects of the school. 

• Frees principal to focus on educational leadership of the school. 

Regional Education Office • Resource link between schools and Central Service Center. 

• Helps to ensure smooth day-to-day operations and provides technical 
assistance where needed. 

Office of Accountability • Oversees and acts as liaison to schools and to support system. 



Source: Chicago Public Schools, 2000a 



institutions of higher education, in 
particular, in the belief that this would 
have a positive long-term effect on 
teacher training. According to district 
administrators, the involvement of these 
external groups would help institutions 
of higher education to better understand 
the challenges and problems facing low- 
performing schools and thus enable 
them to more appropriately prepare 
prospective teachers to work in these 
settings. 



In the next two sections, we delve 
more deeply into the nature of the 
assistance provided by the probation 
managers and external partners. Our 
goal is to describe the variation in 
support and to suggest possible lessons 
for the district and others from this rich 
experiment in assisting low-performing 
schools. First, we lay out the analytic 
frame and methodology we bring to this 
exercise, and then describe the work of 
the providers, using this frame. 
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Summary: Design Features of CPS Support to Probation Schools 

Several characteristics of Chicago's design of external assistance stand out from this discussion. 

• First is the prominence of the support component of the probation policy. Recognizing the 
need for capacity building in low-performing schools, CPS has spent millions of dollars on 
this effort. “ 

• Second is the emphasis on providing site-based assistance in order to improve the school as 
an educational unit. Generic, individually based, off-site professional development activities 
for teachers play little role in the probation policy; rather, teachers receive their professional 
development as part of a school-wide improvement plan, and most of the assistance occurs 
on site with colleagues. 

• Third is the establishment of multiple avenues of support and the separation of the probation 
manager and external partner roles; the former is geared more toward monitoring and 
principal support and the latter toward instructional assistance for teachers. 

• Fourth, the bulk of the external assistance — that of the external partners — is market based. 
Schools select and contract with independent providers. That selection is only moderately 
constrained by the proposal process — directions to partners are very general, and multiple 
approaches and targets are acceptable on the assumption that this flexibility will better meet 
the needs and contexts of individual schools. In addition, monitoring of partners from the 
central office is relatively light as quality and fit are to be controlled in large part by the 
ability of the school to switch partners if unsatisfied.'^ Such switching is facilitated by the 
fact that most partners are based in the Chicago area. 

• A final aspect of the design worth noting is the relative lack of emphasis on training of support 
providers (e.g., probation managers are selected based on their track records and receive 
little or no additional training in mentoring other school leaders) and the absence of built-in 
mechanisms for learning, either on the part of the providers or the district. Partners and 
probation managers report that monthly meetings are mainly for communication from the 
district to the providers, with few avenues for systematically sharing practices or lessons 
from the field. While this is hardly unique to Chicago, the failure to take advantage of the 
natural variation engendered by the market approach may impede improvement of the 
system over time. 



” This approximate figure includes federal Comprehensive School Reform Demonstration program 
grants, which now provide a major source of funding for the external partners. 

“ Indeed, approximately 50% of schools on probation have switched partners at least once. Generally, 
schools initiated the change, but occasionally the partner did so. In either case, a lack of fit is generally 
reported to be the primary explanation, though in some cases, according to the schools in our study, 
partners simply did not follow through on promises made at the time the contract was negotiated. In 
addition, a portion of the switching appears to have resulted from partners "going out of the business" of 
providing support to low-performing schools. 
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Conceptualizing Support: 
Analytic Frames and 
Methodology 

In this section, we draw on several 
lines of research to outline the analytic 
frames for this study. Our discussion 
begins with lessons from the expansive 
literature on policy implementation, 
noting in particular the critical 
mediating role played by agents in the 
intermediate layers of the system, like 
the assistance providers. We then turn 
to the strategies of the providers 
themselves, using literature on 
instructional capacity, school 
intervention, and professional 
development to suggest important 
components of the providers' work. 

Policy Implementation: 

The Mediating Roie of Change 
Agents 

McLaughlin (1987) argues that 
policymakers carmot directly change 
schools because implementation at the 
local level depends on individual as 
well as external factors. Whatever its 
original design, a policy is transformed 
as agents at multiple levels of the 
system interpret, adapt, and act on it 
(Elmore, 1987; Spillane, 2000; Weatherly 
& Lipsky, 1977). This interpretation and 
adaptation are based on two factors: 
beliefs of individuals and the contexts in 
which those individuals work 
(McLaughlin, 1987; Spillane, 2000). 
Beliefs include implicit and explicit 
assumptions about goals, problems and 
their causes, and viable points of 
intervention. Contexts include such 



things as capacity at both the individual 
and organizational levels. Variation in 
beliefs and contexts leads to variation in 
implementation. 

In the case of Chicago school 
probation, the expected variation in 
implementation is likely to be 
exacerbated — even encouraged — by 
reliance on an only moderated, 
constrained marketplace of support 
providers. External partners, and even 
probation managers, bring to the 
schools differing assumptions and 
strategies — or, in the words of Argyris 
and Schon (1978), differing theories of 
action. Moreover, because these 
providers are the ones who work most 
closely with school persormel, their 
theories of action play a crucial 
mediating role for the probation policy. 
The picture is further complicated when 
one considers that there are multiple 
probation support providers for any one 
school (see Eigure 2) and even partners 
and vendors outside the probation 
system. This raises the question of 
whether the theories of action of the 
assistance providers are consistent with 
one another and with those of the 
district administration. Hatch (1998) 
found that differences in the theories of 
action of different school reform 
organizations and individuals may 
inhibit change. A central concern behind 
our investigation of the support system 
in Chicago was to understand these 
differences and their potential impact on 
improvement efforts. 

The implementation literature also 
underscores the importance of 
considering implementation issues at 
several levels of the system — in this 



12 FINNIGAN AND O’DAY 



Figure 3. Implementation Framework 




Note: This framework shows the linkages between the policy and educator performance. However, the policy's 
ultimate goal is to change student performance and the route to bringing about this change is through educator 
performance. 



case, between the district and the 
providers, among the providers 
themselves and between them and the 
schools, and within the schools (as 
teachers and administrators interact 
with the interveners). This report 
focuses primarily on the design and 
implementation of support at the policy 
and provider levels with additional 
supporting data from our case studies. 
Figure 3 portrays this multi-level 
implementation frame for the external 
support component of this policy. Both 
individual actors and the policy context 
affect policy implementation as support 
providers and school staff interpret and 
respond to the policy. An important 
component of this framework is 
represented by the circle (school) nested 
within the rectangle (district) — 



illustrating that the policy targets 
teachers and building administrators 
but that these individuals are nested 
within larger competing contexts. 

Figure 3 also illustrates how the external 
providers assist the school by targeting 
the two levels of capacity: instructional 
capacity in classrooms and 
organizational capacity of the school 
unit. Below, we consider literature that 
guided our analysis of the targets and 
forms of assistance support providers 
pursued in their efforts to build 
instructional capacity. 



External Support to Schools on Probation: Getting a Leg Up? 



13 



Provider Theories of Action: 
Capacity, intervention, and 
Professionai Learning 

As discussed above, providers' 
theories of action and strategies for 
capacity building play crucial roles in 
the implementation and ultimate 
success of the school probation policy. A 
central task of this report is to describe 
the variation in those strategies. A 
variety of literature helps flesh out the 
components of providers' theories of 
action. First, we consider the literature 
on instructional capacity, on the dual 
assumption that the goal of most 
support providers is to increase the 
capacity of schools to improve student 
learning, and understanding how 
providers conceptualize that goal is 
critical for understanding their 
strategies to reach it. We then turn to the 
literature on external assistance and 
intervention as a basis for 
understanding providers' intervention 
strategies. And, finally, we consider 
research on professional development, a 
major component of the providers' 
work. 

Instructional Capacity 

Schools are nested systems of 
individuals operating within 
organizations. Therefore, two levels of 
capacity are important to high- 
performing schools — individual 
capacity and school capacity. 

Goertz, Floden, and O'Day (1995) 
identify four dimensions of individual 
teacher capacity: knowledge, skills, 
dispositions, and views of self. Much of 



the literature has focused on the first 
two dimensions, teacher knowledge and 
skills. In the mid-1980s, Shulman (1986) 
posited that three types of instructional 
knowledge are central to teachers' work: 
knowledge of content (e.g., 
mathematics), pedagogical content 
knowledge (e.g., how children learn 
mathematics), and general pedagogical 
knowledge. This delineation was 
significant in its emphasis on the 
centrality of instructional content in 
teaching, a break from earlier "process 
product" research that focused 
primarily on generic pedagogical 
techniques. Since that time, many 
researchers have explored the role of 
content knowledge and pedagogical 
content beliefs on teachers' work and 
student learning (e.g., Fermema & 
Carpenter, 1992). They have also 
extended the discussion of necessary 
knowledge to include knowledge of 
students and of instructional context.^^ 

Individual educator capacity is not 
enough to ensure student learning, of 
course. Begirming with the effective 
schools research and continuing 
through literature on organizational 
learning and complexity, analysts have 
also focused on the organizational 
aspects of capacity (Levine & Lezotte, 
1990; Mohrman & Lawler, 1996; 
Newmarm & Wehlage, 1995; O'Day, 
Goertz, & Floden, 1995; Purkey & Smith, 
1983). Much of this literature has 
examined the characteristics of high- 
performing schools on the assumption 
that these are characteristics that should 



“ Shulman (1987), himself, noted that teachers 
need other kinds of knowledge including 
knowledge of the curriculum, students, 
educational contexts, and educational ends. 
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Figure 4. Instructional Capacity 




be promoted in less successful ones. 
These characteristics included such 
things as a shared vision focused on 
student learning and common strategies 
for engendering that learning, a culture 
of professional collaboration and 
collective responsibility, high-quality 
curriculum and systematic monitoring 
of student learning, strong instructional 
leadership (usually from the principal), 
and adequate resources. More recent 
research has especially noted the 
importance of professional community 
(McLaughlin & Talbert, 1993; Newmarm 
& Wehlage, 1995) in which information 
and authority are shared (Darling- 
Hammond, 1996; Mohrman & Lawler, 
1996). 

Cohen and Ball (1999) have brought 
together and further complicated the 
two levels of capacity. They argue that, 
at its base, instructional capacity is "a 



function of the interaction among 
[teachers, students, and educational 
materials], not the sole province of any 
single one, such as teachers' knowledge 
or skill, or curriculum." Moreover, 
because the instructional unit (teachers- 
students-materials) is nested within the 
organization, interaction with and 
management of the environment is 
integral to understanding instructional 
capacity. Relevant aspects of the 
instructional environment are the level 
of instructional coordination among 
classrooms and individuals, 
opportunities for professional and 
organizational learning, the prevailing 
normative structures in the school, the 
ways in which teaching and learning are 
monitored and incentives provided for 
improvement, and the level and type of 
resources available. Figure 4 portrays 
Cohen and Ball's conception of 
instructional capacity, to which we have 
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added elements from the literature on 
organizational capacity. This 
conceptualization forms the basis for 
much of our examination of providers' 
theories of action. 

School Intervention 

While educational researchers may 
be able to delineate the characteristics of 
successful, high-capacity individuals, 
units, and schools, much less is known 
about effective intervention strategies 
for developing capacity and improving 
instruction. Indeed, one of the major 
criticisms of the effective schools 
movement was that it assumed a direct 
link between specified characteristics of 
effective organizations and a strategy 
for change. 

Interestingly, despite the millions of 
dollars spent on external change agents 
in school reform (and business 
restructuring), research on their 
effectiveness remains sparse. 
Researchers like Huberman (1995) note 
the importance of bringing new 
information and outside perspectives 
into school communities, but 
investigations like the RAND Change 
Agent Study (Berman & McLaughlin, 
1978) find little impact of external 
change agents, in part because of 
problems of fit between the school and 
the provider and failure of providers to 
adjust their programs to the contexts of 
individual schools. A decade later, this 
finding was modified based on more 
flexible agent-school relationships 
(McLaughlin, 1987), and several other 
studies have found that school 
"coaches" can play an important role in 
developing the capacity of schools 



(O'Day, Goertz, & Floden, 1995), even 
those identified as low performing 
(David, Karmapel, & McDiarmid, 2000). 
But what is the nature of effective 
intervention for improving instruction? 
What are its targets and what are its 
forms? 

Because schools are complex 
organizations, finding effective leverage 
points, or targets, of intervention is far 
from straightforward. Interactions 
among the multiple actors in a school 
setting as well as influences from 
outside make causal attributions in the 
change process difficult (Axelrod & 
Cohen, 1999). Cohen and Ball (1999) 
point out that interveners differ 
markedly in their targets — some focus 
on students, others on teachers, and 
others on curriculum and instructional 
materials. Still others seek primarily to 
alter the environment in which 
instruction takes place by restructuring 
the school organization or adding 
resources. These researchers suggest 
that a change in teachers has greater 
potential because teachers play a 
mediating role in instruction. They also 
argue that interventions that target the 
interaction of the three elements of the 
instructional unit may have the greatest 
effect. 

Whether or not external support 
providers can or will effectively 
influence instructional interaction in the 
classroom, however, is in some doubt. 
Some past research has found that 
external partners are most helpful at the 
school (organizational) level, rather than 
the teacher level, because the external 
partners are unable to give the intense 
assistance to teachers necessary to help 
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them develop skills within their own 
classrooms (Cox, 1983). Similarly, others 
point out that support providers 
generally spend inadequate time in the 
schools and lack a strong vision for 
instructional improvement (Chimerine, 
Haslam, & Laguarda, 1994; Fullan, 

1991). Another limitation of any external 
support is that the partners are 
dependent not only on their own 
strategies and the intensity of their 
assistance, but also on school contexts 
and the behaviors of school staff (Cohen 
& Ball, 1999; Fullan, 1991; Sunderman & 
Nardini, 1999). 

The literature on intervention thus 
suggests several elements for our 
analysis of support providers' strategies: 
the aspects of instructional capacity 
(teachers, materials, students, school 
environment) that external partners and 
probation managers target in their effort 
to improve probation schools, the extent 
to which these represent a clear vision 
and strategy for intervention, and the 
type and intensity of learning 
opportunities they provide to school 
persoimel. These learning opportunities 
are discussed in more detail below. 

Opportunities for Professionai 
Learning 

A key feature of organizational 
capacity is the consistent structuring of 
opportunities for teachers and 
administrators to learn what is needed 
to improve practice. Providing such 
opportunities is one of the central tasks 
of the external partners and, with 
respect to principals, of the probation 
managers. The literature on professional 
development suggests that 



accomplishment of this task, however, 
requires a break with conventional staff 
development models. 

Criticisms of currently dominant 
forms of professional development for 
teachers are well known. Surveys of 
staff development find that it generally 
consists of "unfocused, fragmented, 
low-intensity" activities, such as short- 
term workshops with little or no follow- 
up (Corcoran, 1995). Little (1993) has 
argued persuasively that conventional 
staff development is based on a 
transmission model of learning that is 
inappropriate for current reform goals 
and contexts. 

Meanwhile, studies of effective 
professional development have 
delineated several characteristics found 
to be related to increased teacher 
capacity. In their synthesis of the 
literature, Newmaim, King, and Youngs 
(2000) conclude that "to promote the 
kind of teacher learning that leads to 
improvement in teaching, professional 
development should concentrate on 
instruction and student outcomes in 
teachers' specific schools; provide 
opportunities for collegial inquiry, help, 
and feedback; and cormect teachers to 
external expertise while also respecting 
teachers' discretion and creativity." 
Other researchers have documented the 
relative effectiveness of professional 
development that is focused on the 
content that students are to learn and 
how best to teach it (Cohen & Hill, 1998; 
Corcoran, Wang, & Foley, 1999; 
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Kennedy, 1998)/'* These findings are 
consistent with the recent emphasis on 
content knowledge and pedagogical 
content knowledge in studies of teacher 
capacity. In addition, evidence is 
emerging that this content-based 
professional development must be of 
high intensity and sufficient duration 
before it will have a measurable effect 
on practice (Hawley & Valli, 1998; 
Smylie, Bilcer, Greenberg, & Harris, 
1998). A recent quantitative analysis of 
the National Science Foundation's Local 
Systemic Change initiative, for example, 
found that "it was only after 
approximately 80 hours of professional 
development that teachers reported 
using inquiry-based teaching practices 
significantly more frequently. . .while the 
big change in investigative culture came 
only after 160 hours" (Supovitz & 
Turner, 2000). 

The next section concentrates on 
these elements of support providers' 
strategies in the Chicago school 
probation context. We then consider the 
overall strength of the support system in 
light of our findings and of a framework 
offered by Andrew Porter and his 
colleagues. 



“ Fennema and Carpenter (1992) also found 
significant effects in their work on cognitively 
guided instruction, which focuses on how 
students learn mathematics, as opposed to 
specific strategies for teaching it. 
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Summary of Conceptual Framework 

The literature reviewed above suggests that an analysis of the system of external 

support for Chicago probation schools should incorporate several key elements: 

• Support providers play an important mediating role in the implementation of the 
CPS probation policy. 

• Implementation issues at several levels are likely to cause variation in the provision 
of support in probation schools, variation that is further enhanced by the market 
approach to external partner selection. 

• Because of this variation, it is important to consider the theories of action or 
strategies that providers bring with them into the schools. 

• Central to those strategies are: 

• Providers' conceptions of the problem of low performance and their vision of 
teacher and school capacity. 

• Providers' targets of intervention: specific elements of the instructional unit 
(teachers, students, materials) or of the school's instructional environment 
(coordination and leadership, professional culture, monitoring, and 
opportunities for adult learning). 

• The content, form, and intensity of professional development opportunities 
that support providers offer or sponsor. 
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Provider Strategies and 
Impiementation of 
Assistance 

To summarize, the provision of the 
Chicago school probation policy has 
involved external assistance to more 
than 100 elementary schools that have 
been on academic probation since 1996. 
At minimum, each school worked with 
an external partner and probation 
manager. Most also received input and 
assistance from their regional education 
office, an Office of Accountability 
facilitator, and a business manager or 
intern. 

Despite the universality of assistance 
to schools on probation, the nature and 
intensity of that assistance varies 
substantially from provider to provider 
and school to school. This section 
describes that variation at the level of 
the providers. We begin with a brief 
overview of the assistance provided to 
probation schools. We then describe the 
theories of action and strategies of the 
providers, including the areas identified 
earlier in this report: the providers' 
construction of the problem of low 
performance and goals in working with 
schools, their targets of intervention into 
school organization, and the methods 
and content of their professional 
development. Finally, we consider 
several constraints on the 
implementation of support across 
providers, which serve to limit the effect 
of the assistance on probation schools. 
They also provide the basis for our 
discussion of policy strength and 
implications in the final section of this 
report. 



Overview of Assistance 

During the 2000-2001 school year, 
CPS approved 18 external partners to 
work with probation schools in Chicago 
and 16 of these assisted elementary 
schools.'^ The four largest of these were 
university based and worked in 70% of 
the probation schools. Twelve others — 
three universities and nine independent 
consultants — served five or fewer 
schools. “ 

The background and degree of 
experience of the consultants'' who 
worked directly with schools on 
probation varied by partner. For 
example, the Center for Urban 
Education at DePaul University 
employed young teachers who were on 
loan from CPS for one to two years; 



For this report, we focus on the external 
partners linked to the schools through 
probation, although we have found fhaf many 
schools work wifh a number of ofher exfernal 
agencies to varying degrees. Our case study data 
reveal that the role of fhese addifional providers 
can be very imporfanf in any given school, buf 
for fhe purposes of fhis reporf, we mention fhem 
only vis a vis fheir relationship wifh fhe 
probation supporf sysfem. 

The 11 partners that participated in our study 
are listed m Appendix A. Over the five years of 
fhe policy, six of fhe probation partners have 
either gone out of business or sfopped working 
wifh probafion schools in Chicago: fhe Norfh 
Cenfral Regional Educational Laborafory, 
Momingside Learning Sysfems, Malcolm X 
College, Quality Education Services, Marva 
Collins, and America's Choice. 

'' The external partners send people with a 
variety of fifles mfo fhe schools (e.g., 
implemenfafion specialisfs, coordinators, and 
facilifafors). Eor fhe sake of simplicify, we refer 
to fhese people as consultants. 
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many of them were graduates of the 
DePaul Urban Teacher Corps. In 
contrast. School Achievement Structure 
employed retired teachers and 
administrators, while both Community 
for Learning and Small Schools 
Workshop hired individuals with a 
variety of backgrounds, including social 
workers and psychologists. 

In addition to the various external 
partners, 53 probation managers 
worked with elementary schools on 
probation during the 2000-2001 school 
year. Probation managers are recruited 
by district staff based on their prior 
administrative success within and 
outside of schools. Nearly all of the 
probation managers we interviewed 
described themselves as working 
directly with the principal, and by 
providing ideas, support, and guidance. 
A few reported that their work extended 
beyond the principal to include teacher 
behavior and knowledge. Overall, 
however, probation managers described 
their role in the schools as unclear and 
undefined. 

Construction of the Problem 
and Goals of Support Providers 

The Chicago school probation policy 
derived from a perception that 
persistent low student performance 
stems from problems of the will and 
capacity of the people working in the 
school. In interviews, several district 
administrators attributed school failure 
directly to unmotivated or incompetent 
staff. They described school staff as 
"going through the motions" and 
holding low expectations for students. 

In the words of one district official. 



"Most [of the school staff] are in denial. 
They talk about 'those parents' and lay 
blame without taking any of the 
responsibility." Similar sentiments 
echoed throughout the district-level 
interviews. 

External partners and probation 
managers also locate the problem of low 
performance within the schools — either 
at the classroom or organizational level. 
Two-thirds of the probation managers 
participating in our study mentioned 
problems having to do with low morale, 
inexperience, or other weak aspects of 
teachers at the schools. Half faulted 
weak leadership on the part of the 
principal, either in addition to or instead 
of teacher failings. Additionally, nearly 
all of the external partners thought the 
primary problems in these schools were 
related to poor teacher quality and/ or 
weak principal leadership. By contrast, 
many respondents in the schools point 
to problems outside their control: 
poverty, inattentive parents, and 
dysfunctional communities. This 
disjuncture between support providers' 
views and those of school persoimel is 
hardly surprising. In part, probation 
managers and external partners were 
brought into these schools precisely to 
help the staffs redirect their attention to 
internal beliefs and behaviors that may 
be inhibiting student learning rather 
than focusing on factors external to their 
schools. According to one probation 
manager: 

People had to wake up and start focusing 
on instruction and teaching and 
learning. The schools on probation are 
the ones racially isolated and in low- 
income communities. You don't see any 
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schools on probation in upper-class 
neighborhoods. But in some of the poorer 
communities, you have schools next to 
probation schools that are doing well. So 
there's something dysfunctional within 
the school. . .In many cases, people use 
[external causes] as an excuse, but you 
can do that no longer. 

Interestingly, while both external 
partners and probation managers 
asserted that school staff needed to 
change their beliefs and expectations 
about students, very few mentioned 
such change as among their own main 
objectives, and none seemed to target 
teachers' beliefs about students in any 
systematic way. 

Beyond pointing out the internal 
nature of the problem, support 
providers varied in their description of 
its source or solution. A few focused on 
matters of will, noting that there were 
too many teachers who simply were 
"not teaching." Others acknowledged 
low motivation but attributed it to 
teachers feeling beaten down and 
blamed — in part by the negativity of 
probation reports and media coverage. 
Such determinations affected providers' 
approaches to working with school 
persoimel. For example, Barbara Radner 
of the Center for Urban Education sees 
teachers as "depressed" and therefore 
uses an "asset-based approach," which 
avoids direct criticism of either teacher 
behavior or ideas, instead seeking 
changes through reinforcing their 
positive manifestations. Radner says 
teachers are "individuals who have been 
working hard, are discouraged, and feel 
devalued." 



Whatever the approach to 
motivational issues, probation managers 
and partners concentrated their 
attention most directly on problems of 
capacity. Teachers' knowledge and 
skills were at the heart of the partners' 
assessment of school needs. Sharon 
Ransom of Northeastern Illinois 
University and Barbara Radner of the 
Center for Urban Education, for 
example, both point to the limited 
knowledge base of teachers due to 
inadequate pre-service training. In the 
words of Ransom, "Teachers need to 
have new ideas followed up on and 
time to try out the strategies. . .In some 
cases we are doing teacher training and 
not professional development." 
Consultants with School Achievement 
Structure, by contrast, noted the 
disorganization of instruction, pointing 
out that neither curriculum nor 
individual lessons are well-structured or 
paced. Few external partners identified 
content knowledge as a central problem, 
with the possible exception of Radner, 
who pointed out "elementary [teachers] 
are methods happy. . .We have content 
confusion and methods obsession for 
elementary teachers." Low principal 
skills were also aspects of individual 
capacity noted by partners and 
probation managers. These inadequate 
skills included lack of focus, inattention 
to monitoring of instruction, and poor 
methods of teacher evaluation. 

In addition to low levels of 
individual capacity, all probation 
managers and some of the external 
partners identified larger issues of 
organizational capacity as especially 
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problematic in these schools.^** Common 
criticisms included the absence of a 
coherent vision and a lack of structure 
or coordination across the educational 
program. One probation manager's 
description of the school he worked 
with was consistent with many others' 
comments: "The school was not 
organized, no mission was in place, and 
there was no focus in terms of people 
working together." An external partner 
supported this perspective, saying that 
"teaching and learning issues are 
central, but most of the problems in 
these schools are problems of 
organization and management." 

Intervention Targets and 
Strategies 

Providers' approaches to working 
with individual schools derived from 
their construction of the problem and 
conception of capacity. In this section, 
we describe providers' targets of 
intervention — that is, those groups or 
components of capacity on which they 
concentrated their attention and efforts. 
We then consider the extent to which 
such targeting aggregates into a 
coherent change strategy. 

Intervention Targets 

The external support providers' 
targets of intervention are an important 
aspect of the theory of action. To 
indicate the overlapping nature of 
assistance at the school site, we refer 



“ Four of the 11 external partners identified 
problems at both the instructional and 
organizational levels. These partners are 
described below. 



back to Cohen and Ball's (1999) 
discussion of instructional and 
organizational capacity building. A 
common misconception is that the 
external partners target the instructional 
unit (primarily teachers) and probation 
managers target the environment 
(principals), but in actuality this simple 
division of labor rarely holds. Figure 5 
illustrates the overlapping nature of the 
targets of intervention of probation 
managers and external partners at the 
school site. As seen in this figure, 
external partners focused on both the 
instructional unit and the school 
environment. To the extent that they 
focused on the environment, their 
interventions frequently overlapped 
with those of probation managers, 
especially in the areas of monitoring and 
coordination. 

Targeting the Instructional Unites 

As Cohen and Ball (1999) argue, 
instructional capacity can be improved 
by targeting some combination of 
teachers, materials, or students, and 
especially by focusing on the interaction 
between these three elements of the 
instructional unit. Those external 
partners who viewed low instructional 
capacity as the problem tended to focus 



” This section on building instructional capacity 
will only describe the assistance of external 
partners because probation managers as a rule 
target organizational, rather than instructional, 
capacity. 
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on the classroom, particularly on 
curriculum and instruction. Among 
probation support providers, most 
efforts to build instructional capacity 
focused on either teachers or materials.^” 

Teachers 

According to Cohen and Ball (1999), 
teachers are the most promising points 
of intervention to improve instruction 
because "teachers mediate all 
relationships within instruction." 
Teachers, in fact, were the primary 
target of the external assistance 
provided by external partners, although 



Intervention to improve student motivation 
and capacity more directly took the form of 
policies on sfudenf promofion and retenfion and 
of affer-school and summer supporf for 
struggling students. For the most part, these 
interventions were outside the purview of the 
external partners. 



the specific teachers targeted varied by 
partner. For example, some partners 
limited themselves to working with 
teachers in the accountability grades 
while others worked with every teacher 
in the building. Still others worked only 
with those ready and willing to receive 
assistance. These variations occurred 
within as well as among external 
partners. The approaches seemed to 
have more to do with the degree to 
which individual principals allowed the 
external partner access to teachers than 
with the programs of the external 
partners themselves. In fact, the 
decisions to target particular groups did 
not appear to emanate from a clear 
approach or strategy on the part of 
external partners. 

All of the external partners who 
participated in our study focused on 
developing teachers' knowledge and 
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skills, but few emphasized content 
knowledge and learning in the 
professional development they 
provided. Instead, the external partners 
focused on teachers' pedagogical skills, 
providing tools to help teachers better 
group students and organize their 
lessons. In fact, 8 of the 11 partners 
provided training that emphasized 
generic pedagogical strategies, 
including how to effectively organize 
instruction. For example. School 
Achievement Structure emphasized 
pacing and grouping strategies in their 
"Ten Routines" of effective instruction. 
Similarly, Community for Learning 
trained teachers on how to organize 
students into three ability groups for 
reading instruction. In a 90-minute 
reading block, the teacher spent 
approximately half an hour instructing 
each small group, while students in the 
other two groups worked in learning 
centers or completed independent 
practice. 

Several partners also tried to ensure 
what they considered to be proper 
coverage (e.g., making sure the topics on 
the ITBS [Iowa Test of Basic Skills] were 
taught during the year) by providing 
teachers with calendars and other tools 
for teachers to use to help manage 
instruction. The tools provided by 
external partners were meant to 
guarantee that students were tested on 
the material they had a chance to study. 
In a few cases, the tools were meant to 
expand the instruction beyond ITBS 
coverage. 

Less than half of the partners 
reported developing teachers' skills in 
teaching students how to think critically 



(i.e., teachers' questioning strategies). 
For example. Achieving High Standards 
Project worked with teachers to show 
them that asking particular questions 
could improve comprehension. They 
identified three points when teachers 
should ask questions of students: (1) 
before reading (e.g.. What in my prior 
knowledge will help me with this 
particular task?); (2) during reading 
(e.g.. What information is important to 
remember?); and (3) after reading (e.g.. 
Do I need to fill in any holes in my 
understanding?). Another example of 
this type of approach was the Center for 
Urban Education's focus on having 
teachers ask challenging questions of 
students in book discussions. To remind 
both teachers and students of higher- 
order thinking skills, the Center for 
Urban Education provided a poster that 
summarized the steps to approaching 
reading: (1) Get it (literal questions to 
ask), (2) Get it Glear (analytic questions), 
(3) Think More (inferential questions), 
and 4) Think it Through (evaluative 
questions). Although these strategies 
were more elaborated than others, they, 
too, emphasized the development of 
teachers' pedagogical skills rather than 
their content knowledge. 

A few partners focused on 
pedagogical content skills to some 
extent. For example. Achieving High 
Standards Project provided teachers 
with training and a structure for 
organizing reading instruction based on 
the Four Blocks of Literacy. Achieving 
High Standards Project gave teachers 
forms to plan each lesson around 
guided reading, independent reading, 
word study, and writing. Three of these 
were to be covered in the 90-minute 
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reading lesson at the start of the school 
day; the fourth was to be picked up at a 
later time. The Center for Urban 
Education also provided workshops on 
specific aspects of literacy instruction, 
such as the teaching of poetry, but these 
seemed primarily to provide 
background to the main emphasis on 
generic questioning skills. 

While all of the partners addressed 
some aspect of reading instruction, only 
4 of the 11 partners that participated in 
our study reported having a 
comprehensive reading program.^^ In 
our definition, comprehensive reading 
instruction involves a reading 
curriculum, a consistent and coherent 
set of instructional strategies for the 
teaching and learning of reading, and 
assessment tools for evaluating student 
progress. In addition, professional 
development on reading instruction is 
sustained over a period of time. 
Significantly, the four partners that 
reported this type of comprehensive 
approach served a total of only 10 of the 
elementary schools on probation. In 
addition, across the partners that 
addressed reading to any extent, the 
approach to literacy instruction varied 
tremendously from a focus on fluency to 
word recognition and phonic decoding 
strategies. Rarely did a partner 
emphasize improving teachers' explicit 
understanding of the reading process 



“We found some instances in which external 
partners facilitated a school's relationship with a 
literacy program when they did not provide this 
within their own program or continued to 
support a program that a school already had a 
relationship with. For example, one partner 
brokered the school's relationship with the 
Success for All program. 



and scientific basis of literacy 
instruction. 

Materials 

As illustrated in Figure 4, targeting 
materials is another way to improve 
instructional capacity. Although the 
partners emphasized developing 
teachers' knowledge and skills through 
professional development sessions, 
nearly all of the partners provided some 
type of materials to facilitate the 
implementation of new practices and 
assist teachers in organizing their 
curriculum and instruction. In addition, 
about half of the partners in our study 
requested that schools reallocate their 
resources toward the purchase of 
specific instructional materials. The 
materials the partners provided include 
charts that match ITBS topic areas to 
state and district standards, lesson plan 
guides, "how-to" packets or sheets, and 
assessments. These materials were often 
prominently displayed in the 
classrooms. 

While most partners provided these 
types of tools for organizing instruction, 
few offered teachers actual curricular 
materials for use with students.^ Only 3 
of the 11 external partners provided 
teachers with an actual curriculum, and 
one other partner provided short 
lessons to teachers. Most of the partners 
worked with the existing basal reading 
series. Approximately half of the 
partners provided teachers with 



“ While partners did not tend to provide this 
t5q)e of material, the district recently supplied 
structured lessons as a resource to teachers. 
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assessment tools to be used regularly in 
their classrooms. These tools focused on 
discrete skills assessed on the ITBS. 

Students 

Another important strategy for 
improving the instructional unit is to 
intervene directly at the student level. 
Thus, alongside school probation, CPS 
initiated various programs and policies 
to target students in these schools. 
Principal among these were the student 
promotion policy, the Lighthouse 
Program, and summer school. The 
external partners' work, however, was 



intended to focus on the adults and it 
did so almost exclusively. 

Some schools contracted with other 
providers to work directly with 
students. For example, several probation 
schools contracted with Success Lab, a 
private company that provides students 
with extra assistance in reading once a 
week. During these pullout sessions, 
trained coaches worked with small 
groups of two-to-four students. 

Students read books at their 
achievement level and, at the end of 
each session, completed a computerized 
test that marked their progress. Based 
on this test, the coach individualized 



A Variety of Materials for Schools 

Several partners provided teachers with materials to assist them with new curricular and 
instructional strategies. The following examples illustrate the variety in the types of 
materials provided to school staff. 

Accountability Direct Instruction Model and the Brad Frieswyk Group emphasized direct 
instruction of scripted lessons. These partners provided a variety of materials to educators: 
charts to track reading fluency, protocols to help students build comprehension skills, and 
direct instruction books and lessons. 

School Achievement Structure had educators follow a set of "structured routines" around 
discipline, instructional pacing, and assessment of skills. These routines are grounded in the 
effective schools research. School Achievement Structure asked teachers to complete skills- 
pacing charts that document whether or not each skill tested by ITBS has been learned "to 
mastery levels" — teachers indicated whether students had been introduced to a skill, and 
when they had reached proficiency. This partner also connected its schools to a software 
company that provided them with a bank of test items they could draw from for grade-by- 
grade assessments. 



External Support to Schools on Probation: Getting a Leg Up? 



27 



the program to meet the student's 
needs. Success Lab's contracts varied by 
school/'’ but the company guarantees all 
schools a refund if they do not see an 
average of one year's growth across 
students. 

Interaction among Teachers, 
Students, and Materials 

A couple of the partners targeted 
teachers and materials together by 
training teachers in the use of certain 
types of materials in the classroom. To 
the extent that the partners worked with 
teachers directly in their classrooms, 
they may have also focused, at times, on 
the interaction between teachers and 
students. However, the interaction of 
students, materials, and teachers was 
not at the heart of what these partners 
did. For example, little analysis occurred 
with teachers around student work and 
the kinds of instruction that produced 
the work. We did observe one exception 
to this pattern: teachers who 
participated in the Center on Urban 
Education's Saturday workshop shared 
their students' work around a curricular 
or instructional area that the teachers 
learned about the previous week. For 
example, a middle school science 
teacher brought books her students 
made about rainfall and its 
consequences to the workshop to 



^ The contracts tend to cost $80,000 to $100,000. 

“ In our case study schools that employed 
Success Lab, however, we saw no attempt to 
track its impact on student achievement /test 
scores. We also found no evidence of any 
aftempf fo coordinate the work of these other 
agencies with those of the external partners (or 
with the general education program of the 
school, for that matter). 



discuss with other teachers after they 
learned about expository writing in a 
previous workshop. However, our data 
provide no indication of the regularity 
or frequency of these activities. It is 
conceivable that external partners' 
direct mentoring and observation of 
teachers in classrooms involved more 
emphasis on the interaction of these 
three elements, although neither 
interviews nor observations indicated a 
regular pattern in this regard. 

Targeting the Instructional 
Environment: Building 
Organizational Capacity 

As illustrated in Figure 4, Cohen and 
Ball (1999) also emphasize the 
importance of managing the 
environment around the classroom or 
basic instructional unit. In the probation 
system of support, the work of the 
probation managers and that of the 
external partners overlapped in terms of 
the organizational or environmental 
level of the school. Six of the 11 partners 
and all of the probation manager 
respondents focused on the internal 
school environment or organization to 
build capacity. The probation managers 
in our study focused exclusively on 
organizational capacity — by design, 
their role is to "mentor" the principal, 
usually in the areas of management and 
leadership. In addition, two partners 
focused exclusively on organizational 
structures, leadership, and 
management, often targeting the 
principal or leadership team. The 
remaining four of the six partners that 
targeted organizational capacity did so 
in conjunction with their efforts to target 
the instructional unit itself. 
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Based on our conceptual framework, 
we analyze the organizational efforts of 
external partners and probation 
managers along four dimensions: 
coordination and leadership, 
professional norms for teachers and 
teaching, resources for learning and 
instruction, and the monitoring of 
instruction and student learning. This 
section describes the variation in, and 
overlapping nature of, the 
organizational capacity interventions at 
the school site. 

Coordination 

The support providers' focus on 
coordination of school-level factors 
primarily involved working with the 
principal and/ or leadership team to 
strengthen the school's leadership and 
to develop a shared vision and common 
goals across the school's staff. Most of 
the work of the probation managers 
targeted coordination as they worked 
one-on-one with principals, mentoring 
them by providing ideas and guidance 
and developing their management 
skills. One probation manager described 
his role as follows: "I see myself as a 
coach, as a devil's advocate, as a 
brother. I make sure they're doing what 
they're supposed to be doing." 

Similarly, School Achievement 
Structure, Center for Urban Education, 
and Northeastern Illinois University 
focused on strengthening the 
management and instructional 
leadership skills of the principal. 

A common view was that the 
principals needed to become stronger 
leaders and, to encourage their 
development, both probation managers 



and partners recommended generic 
strategies to the principals. For example, 
providers recommended that principals 
become more visible (in the hallways 
and classrooms), that they monitor 
instruction more frequently, and that 
they follow through with teachers on 
instructional issues. One external 
partner, the Valina Miller Group, 
identified tensions and divisions among 
staff and leadership as a major barrier to 
improvement in at least one of its 
probation schools, and hoped to 
encourage the school to work more 
collectively as a team. 

Another strategy related to 
coordination at the organizational level 
was to assist the schools in developing a 
common vision and to help them in 
narrowing their goals and strategies to 
support this vision. The primary 
example of this strategy was 
Northeastern Illinois University's 
approach, which provided intensive 
assistance by working with teachers and 
administrators at a school site to 
develop a set of common goals and a 
school-wide vision. Northeastern Illinois 
University saw this as a key leverage 
point for school improvement. 

Similarly, 3 of the 11 partners and 5 
probation managers reported working 
with the school on the planning process, 
but this was done on a much less 
intensive scale than the needs 
assessment and strategizing work of 
Northeastern Illinois University. For 
example, some of the partners helped 
the schools by developing and 
administering surveys to inform their 
plaiming for the following year. 
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Helping Schools Identify Needs and Focus 

To begin a partnership with a school. Northeastern Illinois University conducted a 
collaborative needs assessment. After meeting with the principal and the school's 
leadership team to discuss visions and belief systems. Northeastern brought the entire 
faculty together to meet with its team of 8-10 faculty members and consultants. In small 
groups of six-to-eight teachers, the Northeastern facilitators posed one significant question: 
"What in your opinion could you/we do that would increase student achievement?" The 
facilitators asked questions and got everyone's input. Heather Patay, Northeastern's 
director, described the process: "We go around a few times on the question. As time 
progresses, you get layers of responses. Then we come back and ask people to choose their 
three-to-five top priorities." She believed that one of the advantages of this approach was 
that it respects the strengths of the school. The work plan that was developed from this 
process allowed Northeastern to build on what was happening at the school. "This is the 
first step in dialogue and building trust," said Patay. "This way, we prioritize what's 
important." 



Professional Norms 

Some managers and partners also 
targeted one of the most difficult areas 
to change in schools: professional 
norms. The two institutionalized norms 
of particular concern were the isolation 
of instructional practice and the 
inadequate use of data. A handful of 
support providers (both probation 
managers and external partners) 
mentioned trying to change teachers' 
beliefs about students (i.e., their low 
expectations), but systematic efforts in 
this direction were difficult to detect in 
our observations and interviews.^® As 
mentioned previously, an overall goal of 
the external support system was to 
facilitate the school staff's internal 
examination of problem areas 
contributing to the school's low 



“We detected no regular efforts in this regard 
either in our interviews of probafion manager 
and external partner respondents or in our 10 
case study schools. 



performance. However, as one 
probation manager observed, they were 
not trained in working with the schools 
to facilitate this process. 

Several external partners targeted 
organizational structures that, once 
implemented, would facilitate teachers 
sharing ideas and practices, rather than 
teaching in isolation. A common 
approach recommended by the external 
partners was the establishment of grade- 
level teams or clusters. Two probation 
managers also encouraged more 
interaction and discussion among 
teachers, with the goal of having them 
share their ideas and practices with their 
colleagues. Such structures and 
practices have potential not only for 
fostering learning, but also for better 
coordination of instruction throughout 
the school. One probation manager, a 
sitting principal, created an exchange 
program among his teachers and the 
probation school's teachers. 
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In addition, the partners and 
probation managers reported efforts to 
develop educators' knowledge and 
skills around the use of student data to 
inform classroom instruction. Two 
partners. School Achievement Structure 
and Community for Learning, collected 
and analyzed data to determine student 
progress and, subsequently, reported 
their findings to the schools. In addition, 
four partners encouraged schools to 
focus more on their student 
performance data, but did not provide 
intensive assistance to the schools so 
that they learned how to analyze the 
data without the help of the partner. 
While the external partners emphasized 
the use of data in the classroom — to 
help teachers determine ability group 
placements and level of mastery — a 
few probation managers encouraged 
administrators to use the data as a 
monitoring tool and part of the plaiming 
process, as discussed in more detail in 
the section below on monitoring. We 
found little evidence of communication 
between partners and managers about 
these overlapping strategies regarding 
the use of data. 

Resources for Learning and 
Instruction 

Another way in which the partners 
and managers leveraged change at the 
organizational level was by garnering 
and brokering resources for the schools. 
In some cases, this began with attention 
to resources currently in place in the 
school — primarily persoimel. One 
School Achievement Structure 
consultant said, "I also go to the 
principal and ask to see the 
qualifications of his teachers. I might 



find that a first-grade teacher has no 
primary certification. I'll ask, 'Why 
don't you get someone in there who has 
the proper certification? Or make that 
teacher aware they need to go back to 
school.'" A few of the probation 
managers went as far as identifying 
teachers whom they thought the 
principal should replace. 

Both partners and probation 
managers reported they informally 
responded to a school's identified needs 
by referring the staff to appropriate 
organizations to assist them or by 
helping the principal navigate the 
district's system. For example. 
Northeastern Illinois University 
coimected teachers to the library of 
teaching materials available at the 
Chicago Teachers' Center. In addition to 
brokering resources, more than half of 
the external partners pushed schools to 
reallocate resources to pay teachers 
stipends, to purchase materials, or to 
hire new persoimel. When reallocation 
was not possible, the partners appealed 
to CPS or another organization for 
funds to implement these changes. 

At times, the partners brokered 
services because of limited capacity 
within their organizations. For example, 
several partners had their schools 
subcontract with an independent 
organization to assist with the writing 
and scoring of five-week assessments. In 
some cases, when the brokered services 
were monitored and coordinated by the 
external partner, they appeared to be an 
effective means of extending the 
partner's capacity. In others, however, 
multiple partners appeared to 
contribute to a fragmentation of support 
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and even conflicting messages or 
processes. We discuss this problem 
under implementation constraints 
below. 

Monitoring of Instruction and 
Student Learning 

Effective use of information is a 
central feature of high-performing 
organizations (Mohrman & Lawler, 
1996). External partners and probation 
managers worked with school staff on 
the monitoring of information at the 
school level in three areas: student 
performance, classroom instruction, and 
implementation of improvement plans 
and programs. 

Erequent monitoring of student 
performance is a characteristic of 
"effective schools" (Edmonds, 1979). As 
mentioned above, external partners and 
probation managers worked with 
schools to increase the use of data in 
monitoring overall school progress and 
making decisions about school 
improvement plaiming. Eor example, 4 
of the 11 external partners discussed the 
results of ITBS data with school staff to 
identify the collective strengths and 
weaknesses of students. None of the 
partners or managers reported 
analyzing state assessment data, 
although this is not that surprising since 
probation is based on the ITBS, not the 
IGAP (the Illinois Goal Assessment 
Program). 

A few external partners and most of 
the probation managers emphasized the 
importance of increased monitoring of 
instruction by the principal. They 
encouraged principals to visit 



classrooms to check that teachers were 
actively teaching and were using the 
designated materials. They also 
supported the regular review of lesson 
plans. Information on instructional 
practice was generally less systematic in 
either its collection or interpretation 
than were student achievement data, 
however. Eor the most part, partners 
and probation managers seemed to 
focus on the principals checking to see 
that teachers were teaching and that 
there was at least minimal compliance 
with the approved instructional 
program. 

Both probation managers and 
external partners monitored the 
implementation of the reform process in 
the school, including their own 
programs and of the school plaiming 
process. Although monitoring the 
School Improvement Plan for 
Advancing Academic Achievement is 
an area that falls under the district- 
specified responsibilities of the 
probation manager, only one-quarter of 
the probation managers interviewed 
mentioned involvement with this 
planning process. More frequent 
monitoring of overall school 
performance occurred through the 
regular evaluations that the partners 
and managers provided to GPS. Three of 
the 11 partners evaluated the degree of 
implementation at the school site 
through site visits to observe classroom 
instruction and discussions with school 
staff about their improvement efforts. 
Neither partners nor probation 
managers reported having 
consequences or strategies in place that 
would be implemented when scores 
were not increasing or when schools 
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were not improving by some other 
measure. 



schools — only 1 of the 16 probation 
managers involved in our study cited 
problems or concerns with an external 
partner. Two probation managers had 
recommended that their schools select 
new partners, basing these 
recommendations on their view that the 
"fit" was bad, not that the partners' 
programs were weak or misguided. 



The probation managers' informal 
role of monitoring the external partners 
was not clearly defined but was 
mentioned in district and probation 
manager interviews. For the most part, 
the managers felt that the partners were 
providing much needed support to the 



Evaluating Program Implementation 



Two external partners developed strategies to determine the degree of implementation of 
their program and/or the school's improvement plan at each school. The external partners 
followed up after the visit, identifying weak areas of implementation. 

The School Achievement Structure Impact Visit took place over a two-day period, 
wherein several School Achievement Structure consultants observed classrooms to gauge 
how well the school was implementing the improvement plan and the organization's Ten 
Routines. The group wrote a report and then met with the school's leadership team to 
address the findings and design remedies. The School Achievement Structure team 
returned at a later date to determine whether or not the school had corrected the identified 
problem areas. Its goal was for their schools to internalize this process and become 
responsible for their own monitoring. 

The Grand Rounds Review was one component of the Center for School Improvement's 
quality-assurance system. The process began with schools completing a self-study in three 
areas: literacy, social services, and leadership. Teachers reflected on their use of the literacy 
framework in classroom practice. This self-study was combined with an observation by the 
school's literacy coordinator. The two sources of data were aggregated across all teachers 
and provided a profile of the depth and quality of the implementation of the literacy 
initiative. Center for School Improvement staff and the school's leadership used these 
findings to review school operations and plan for the future. 
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Change Strategies 

As is evident from the preceding 
discussion, external partners and 
probation managers targeted multiple 
aspects of instructional capacity. The 
ultimate question is whether these 
targets added up to effective strategies 
for changing the organization and 
improving instructional practices 
throughout the school as a whole. Put 
another way, how strategic was the work 
of the external partners and probation 
managers overall? How did these 
groups attempt to leverage, disseminate, 
and sustain more effective practice? 

Unfortunately, we found that most 
of the probation managers and external 
partners lacked a strategic approach to 
bringing about school- wide 
improvements. Few had well- 
articulated strategies, targeting instead a 
variety of different problem areas at the 
school and/ or the classroom level — 
apparently on the belief that small 
changes in these areas would eventually 
cumulate over time. A few of the 
external partners, however, had thought 
through a more coimected approach for 
leveraging organizational change or 
disseminating new practices. One such 
strategy focused on collective vision- 
setting and the brokering of services 
consistent with that vision (a classic 
organizational development approach) 
on the theory that collective buy-in was 
the key element to successful change. 
Another strategy emphasized 
strengthening the leadership structure 
of the school on the belief that 
monitoring and controlling teacher 
behavior was key. With respect to 
instruction and the dissemination of 



more effective instructional practices, 
most partners relied on general 
workshops and fairly ad hoc 
observations of individuals. 

Two more systematic strategies were 
evident among a handful of partners: 
the train-the-trainer approach and on- 
site literacy coordinators. The train-the- 
trainer approach involves a few school 
persormel attending workshops and 
then being responsible for enlightening 
their colleagues at the school site. Our 
observations in the case study schools 
suggest that this model suffered not 
only from the usual dilution of such 
approaches but also from inadequate 
preparation of the trainers themselves. 

A more promising strategy appeared to 
be the provision of a well-trained, full- 
time on-site coordinator in particular 
curriculum areas — primarily reading, 
as practiced by Success for All and the 
Center for School Improvement. These 
individuals were better prepared in the 
content areas than is usual in train-the- 
trainer approaches, and they held a 
structural position in the school that 
allowed them to work more closely and 
consistently with staff over an extended 
period of time. This approach is 
consistent with the literature on 
effective adult learning as discussed in 
this report. 

Opportunities for Aduit 
Learning: Methods and Content 
of Professional Deveiopment 

Whatever the leverage points for 
organizational change, all probation 
managers and external partners are 
responsible for providing professional 
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development opportunities to teachers 
and/ or administrators at the school site. 
Our review of the literature on effective 
professional development emphasized 
that it should be focused on both 
content and pedagogy; that it provide 
opportunities for inquiry, help, and 
feedback on actual practice; and that it 
be of sufficient intensity and duration. 
Conventional reliance on short-term 
workshops isolated from practice has 
been particularly criticized. 

Unfortunately, the majority of 
external partners' assistance continued 
to reflect the conventional model of 
fragmented, one-shot professional 
development opportunities. This 
conventional professional development 
frequently took the form of half-day and 
full-day workshops and training 
sessions occurring either during the 
school day or on weekends.^*^ Six of the 
11 partners (working with 61 schools) 
primarily provided this type of 
professional development, mostly 
during restructured days. The topics of 
those sessions were diverse, ranging 
from implementing learning centers to 
teaching reading. A few of these 
partners focused staff development on 
helping teachers understand the value 
of grouping students for instruction 
(e.g., ability grouping) within each class 
and/ or grouping students for re- 
teaching of particular concepts not 
mastered on frequent assessments. We 
found little evidence of partners having 
teachers reflect on how the professional 
development offered in workshop 
formats would (or subsequently did) 



^ Three partners (working with 19 schools) offer 
week-long workshops. 



influence their individual classroom 
practice. 

Beyond conventional professional 
development, all partners reported 
working with individual teachers, 
primarily by modeling behaviors and by 
observing and providing feedback on 
teachers' classroom instruction. Partners 
reported "observing" teachers in their 
classrooms, although this activity 
apparently ranged from formal 
observation to an informal walk- 
through. In the course of their 
observations, the consultants reportedly 
focused on how well teachers 
implemented the routines, practices, or 
strategies from the workshops. An 
example of this type of assistance 
observed during the study involved a 
consultant working side-by-side with 
the teacher on a direct instruction 
lesson. The consultant began by 
modeling the instruction; he then 
observed the teacher implementing the 
same step. 

The one-on-one assistance provided 
by partners varied in its frequency and 
intensity. Some of the consultants 
reported observing and providing 
feedback to teachers each time they 
were on site, but the number of teachers 
receiving this type of assistance was 
difficult to determine. For example, one 
partner reported that its consultants 
assisted four teachers a day, on average, 
but what exactly this assistance entailed 
and how much time was spent with 
each teacher was not clear. Nor was it 
clear how often the consultants followed 
up on those visits. In our case study 
schools, for example, we found that 
many consultants spent about a day 
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each week at the school going from 
classroom to classroom, observing 
teachers, providing feedback, and 
distributing materials. Unfortunately, 
the lack of consistent follow-up to these 
individual sessions was a common 
theme among the teachers in these 
schools — presumably because the 
consultant's time was spread across a 
large number of teachers at the site. 
Support providers had various 
approaches to rationing their time and 
assistance to respond to this issue. In 
many cases, they chose to focus 
primarily on the teachers who requested 
assistance, while perhaps stopping by 
the classrooms of other teachers to see if 
they needed help. In a few others, 
consultants had more scheduled 
approaches. For example, one 
consultant worked in the school one 
week every month, spending a day each 
with the five grade-level teams in the 
school. 

Although most partners reported 
providing at least some one-on-one 
assistance as part of their programs, we 
found that the purpose of that 
assistance, which mostly was in the 
form of modeling behaviors, was not 
always clear to the teachers. In some 
cases, the assistance appeared to serve 
as a way for consultants to prove their 
legitimacy to school staff, especially 
initially. As one consultant reported, "In 
September and October, we model. We 
meet later and talk about it. Fifty 
percent of it is they're evaluating you. I 
want them to say, 'I need to try this.' 
They're not understanding that 
modeling is for [them] to do it." 



Probation managers also used 
observation and modeling techniques 
with principals. Like that of external 
partners with teachers, however, 
probation managers' assistance to 
principals varied and was generally of 
low intensity. Probation managers 
reported that they met with principals 
(at most) a few times a month to help 
them with leadership, management, and 
monitoring of the school. A couple of 
external partners (but no managers) 
facilitated the development of 
principals' knowledge and skills 
through networks in which principals 
had the opportunity to share ideas and 
get advice. These networks also 
introduced the principals to research, 
allowing them to strengthen their bonds 
to other principals and receive 
encouragement in their school 
improvement efforts. 

Beyond standard professional 
development opportunities, 3 of the 11 
partners offered courses for which 
teachers received college credit. For 
example. Accountability Direct 
Instruction Model implemented an 
evening class for teachers to deepen 
their understanding of how to use direct 
instruction techniques in the classroom, 
and the Center for Urban Education 
offered a 12-week course for teachers 
that focused on reading instruction. The 
Valina Miller Group customized a 
semester course for interested teachers 
in one of its schools. Teachers selected 
the topics to be covered and received 
college and continuing education credits 
for their participation. Topics tended to 
be discoimected, however, and the 
"course" syllabus took on the 
appearance of a series of workshops 
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rather than an integrated investigation 
of a particular area. Nonetheless, these 
courses were more intensive 
opportunities for teachers to learn about 
new curricular and instructional 
strategies with other teachers from their 
schools. Unfortunately, we do not have 
data on their ultimate effectiveness. 



Implementation Constraints of 
External Assistance 

All of our evidence points to the fact 
that probation schools were universally 
receiving assistance from probation 
managers and external partners. The 
variation in strategies that we have 
described is consistent with what we 
would expect in a generally market- 



An Extended Class Asks Teachers to Explore New and Current Practices 

Jean Spencer is a 26-year veteran of CPS who retired eight years ago and became a reading 
consultant for DePaul University's Center for Urban Education after teaching early literacy 
workshops across the country. Her experience as a consultant to probation schools led 
Spencer to believe that teachers "need to spend more time looking at the latest research for 
new ideas and validating what they're doing and then sharing it with each other in a more 
structured way than in common planning." At Pickard Elementary, she designed a 12-week 
course on rethinking approaches to literacy instruction. The course, which met for two 
hours on Monday afternoons and was taken for credit, had three elements: independent 
reading, discussion and lecture, and a sustained inquiry by teachers. Throughout the 
course, she asked teachers to be reflective about new and current practices. Spencer said, 
"My thing is, 'Why am I doing this?' Is it best practice? Who says it is? How do we know 
it's best practice? What are your experiences with it?" 

She began each class with 30 minutes of reading time, during which teachers selected from 
professional books and journals. In essence, she was modeling the "Drop Everything and 
Read" technique. The teachers kept a double-entry journal on their readings, indicating 
what they read and how they planned to use it in their classrooms. Next, Spencer would 
lead a discussion around recent research (e.g., reciprocal teaching) in which teachers and 
students take turns discussing a text. She would supplement the readings and lectures with 
videos of classroom lessons and concluded each class with a focused inquiry in which 
teachers chose a topic, generated questions, and answered them. At the end of each class, 
teachers completed an exit card indicating what they learned, how they planned to use it, 
and what they still needed to know. To follow up, Spencer was present in the building one 
day a week to meet during common planning times. She reported that teachers were more 
actively seeking out new ideas grounded in research, giving them a deeper understanding 
of what students need to become good readers and writers. 
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based support system. In addition, the 
parties interviewed for this study 
reported general satisfaction with the 
support system.^^ Random samples of 
probation principals rated both external 
partners and probation managers fairly 
highly, although probation managers 
were viewed slightly better than 
external partners. One complaint the 
principals had about partners was that 
they did not appropriately adjust their 
program to the school site. Similarly, 
nearly all of the probation managers in 
our study had good things to say about 
the partners; only one thought there was 
too much paperwork and another 
expressed concern about a particular 
partner.^** External partners, however, 
were somewhat more critical of both the 
principals and the probation managers, 
expressing frustration that the former 
did not always follow their programs 
and the latter were not involved enough 
with their schools. 

Despite the apparent mutual 
satisfaction, our analysis suggests 
several problem areas that we believe 
will limit the impact of this support 
system. These areas, while overlapping, 
fall into the following five categories: 

. Low levels of intensity; 

• Lack of communication among 

providers; 



We should note here, however, that comments 
from teachers in our case study schools are 
considerably more varied in their estimation of 
fhe supporf providers in fheir schools. 

Two probation managers complained abouf a 
former exfemal parfner fhaf is no longer 
working wifh probafion schools. 



. Uneven targeting of literacy 
instruction; 

• Lack of clear strategies for 
organizational change; and 

. Reliance on traditional professional 
development strategies. 

Each of these is discussed in more detail 
below. 

Low Levels of Intensity 

While varying among providers, the 
intensity of assistance delivered to 
probation schools is, we believe, simply 
inadequate to address the substantial 
needs of the individuals and 
organizations concerned. To determine 
the intensity of the support, we 
estimated the number of person days 
per month that the partners and 
managers worked (in some way) with 
the schools. On average, external 
partners spent one-to-two person days a 
week assisting each school (which was 
then divided among attention given to 
individual teachers).^'’ Eor example, two 
partners had consultants in the building 
two days a week but one sent a general 
consultant for two whole days, while 
another sent a math and a reading 
specialist, who spent two half-days 
each. Similar to external partners, the 
amount of time probation managers 
spent in the schools varied. Most of the 
probation managers reported being in 
the school two-to-three times a month, 
with the rest at the school once a month 
or less. A few managers mentioned that 



Some of fhe partners have changed the level of 
intensity over time. 
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their involvement was more intense at 
the beginning of their relationship with 
a school, but had lessened over time. 
This is consistent with our case study 
data, which suggest that while 
assistance from the probation manager 
may have been salient in the initial 
stages of the probation process, it had 
become less so by the second or third 
year of probation. The turnover of 
probation managers at the sites also 
tended to fragment and weaken their 
assistance. Finally, both partners and 
managers reported scaling back what 
they had originally plaimed because of 
limited time and resources and bigger 
needs at the school level than 
anticipated. 

The variation in intensity of the 
external partners' support was 
originally coimected to cost, but this is 
no longer the case because most of the 
partners' services now cost 
approximately $45,000. In addition, 
during our study, 41 of the probation 
schools received federal grants from the 
Comprehensive School Reform 
Demonstration program that covered 
the entire cost of this support. Two 
partners. Northeastern Illinois 
University and School Achievement 
Structure, lowered their original fees of 
$90,000 to $100,000 a year to the same 
level as the rest of the schools. This shift 
to a common amount charged for 
services seemed to coincide with the 
schools paying for the support out of 
their own budgets and the 
Comprehensive School Reform 
Demonstration grants becoming more 
widespread. 



Some partners attempted to extend 
the reach beyond the time that the 
consultants were in the schools through 
train-the-trainer approaches and on-site 
literacy coordinator models.^” Our 
observations in case study schools 
suggest that professional development 
in the train-the-trainer model was much 
too weak to extend into other teachers' 
practices. On-site literacy coordinators, 
however, such as those trained through 
the Center for School Improvement, 
received extensive development in 
literacy instruction and remained on site 
on a full-time basis to assist teachers and 
coordinate and monitor instruction. 
Their roles as instructional leaders in the 
schools seem to be a promising 
approach for enhancing organizational 
and instructional capacity of schools. 

One apparent consequence of the 
limited time most external partners 
were in the schools, coupled with the 
demands for immediate improvement 
on the test scores, was an impatience 
among some external partner 
consultants such that the assistance 
devolved from "capacity building" into 
what we have termed "capacity 
substitution." In theory, the external 
partners worked with school leadership 
(e.g., observing and monitoring classes 
and offering feedback to teachers). 



™ Three of the schools in our 10 case studies 
used on-site literacy coordinators or coaches, in 
one school, the coordinator was an integral part 
of the literacy program. In another, the literacy 
coordinator was cormected with Success for All 
and worked half-time to provide support — an 
intervention brokered by, but not provided by, 
the school's external partner. In the third school, 
the school hired one person to work part-time as 
a literacy coordinator. This hiring was unrelated 
to the work of the external partner. 
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However, what we found in many cases 
was that the external consultants did 
this work themselves. For example, in 
three schools, rather than guiding the 
assistant principals in how to run grade- 
level meetings, the consultants served as 
substitutes for these staff and led the 
meetings themselves. The assistant 
principals, relieved of this 
responsibility, attended to other duties. 
While this enabled administrators to 
respond to day-to-day issues as they 
emerged, when the school reduced or 
ended its partnership, there was no one 
to take on this important task. Similarly, 
several partners wrote the reading plans 
for their schools. Rather than using this 
as an opportunity for the school to 
adapt the reading instruction program 
to suit its individual needs, the partner 
supplied the vision and program to the 
school. 

Lack of Communication among 
Providers 

The lack of communication among 
providers (external partners with other 
partners and probation managers with 
other managers) is noteworthy. Both 
external partners and probation 
managers met with district 
administrators, but these meetings 
appeared to be focused on 
administrative procedures, rather than 
on the sharing of successes and failures. 
Perhaps more important, however, was 
the need for communicating between 
these two types of support providers, 
especially when the partner intervened 
in the environmental level of the 
organization. Unfortunately, we did not 
observe or hear evidence of probation 
managers and external partners actually 



collaborating, or even communicating, 
about their overlapping strategies. 

Beyond official probation partners, 
multiple other external agents provided 
assistance to these schools. 
Communication among these agents 
and probation partners was at even 
lower levels, a situation we expect may 
limit the effectiveness of the policy's 
external support system. When various 
groups involved in the same effort have 
contradictory theories of action, the 
group's effectiveness is severely limited 
(Hatch, 1998). While in the probation 
schools the efforts of managers and 
partners are not contradictory by 
design, the existence of multiple groups 
heightens the probability that school 
staff will be forced to contend with 
conflicting messages about school 
improvement. In fact, half of our case 
study schools had multiple partners 
with overlapping strategies. As a result, 
opportunities for real school 
improvement were duplicated, 
contradicted, or lost. 

Uneven Targeting of Literacy 
instruction 

One rather surprising finding, both 
from our interviews with support 
providers and our observations in case 
study schools, was that the support 
providers, overall, implement strategies 
that only weakly attempt to improve 
literacy instruction. More than half of 
the partners implemented uneven and 
low-intensity strategies that were not 
comprehensive attempts to improve 
teaching and learning around literacy. 
This finding is problematic considering 
schools are placed on probation as a 
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result of low reading scores on the ITBS 
and an implicit understanding at the 
district level is that the support 
provided would improve reading 
achievement. The wide-ranging 
problems or issues that the support 
providers identified in the schools, from 
the facility to leadership to instructional 
capacity, resulted in a weak 
instructional focus. While the external 
partners and probation managers 
tended to develop generic strategies to 
improve the instructional unit and the 
environment, these strategies were not 
directly linked to literacy. 

An additional issue at the school site 
was that approximately 50% of schools 
on probation had switched partners 
over a five-year period, causing the 
school staff to constantly adapt to new 
programs with different curricular goals 
and interventions. For example, one of 
our case study schools switched 
partners, changing from a partner with 
a constructivist approach to one with a 
direct instruction philosophy. In our 
case study schools, we also witnessed a 
high level of turnover of consultants at 
the school site (within partners). At 
times, this had the same effect as 
switching partners as consultants 
identified various targets of intervention 
rather than solely focusing on literacy 
instruction. 

Lack of Clear Strategies for 
Organizational Change 

While all of the probation managers 
and some of the external partners 
targeted the environmental level of the 
school organization, they did not 
articulate a comprehensive intervention 



strategy based on a theory of action. 
Most of the external partners that 
focused on this level of the organization 
did not integrate this intervention with 
their instructional strategies, leading to 
a fragmented approach to change. The 
probation managers implemented a 
more narrow approach to school 
improvement, focusing solely on the 
environmental conditions. Not 
surprisingly, however, expertise as an 
administrator did not automatically lead 
to the ability to transform a low- 
performing school from the outside. In 
fact, improving the performance of 
these schools appeared to have been 
more difficult than most probation 
managers expected. They expressed 
frustration with the slow progress, 
reporting that unreceptive, distrustful 
staff sometimes disregarded their 
recommendations . 

Reliance on Traditional Professional 
Development Strategies 

A final area of concern is the reliance 
on traditional professional development 
strategies — primarily workshops — 
with little intensive assistance to 
teachers. This type of low-intensity 
assistance has been found to have 
limited or no effect on teaching and 
learning (Supovitz & Turner, 2000). 
Although substantial resources from the 
schools and district were allocated 
toward external support, the level of 
intensity necessary to improve the 
instructional capacity of such a large 
number of probation schools may, in 
fact, be difficult to attain. At the very 
least, content-specific professional 
development is critical to improving the 
knowledge and skills of teachers around 
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literacy instruction. In the current 
system of support, the reliance on 
uncoordinated professional 
development opportunities, which were 
not sustained over time, limited the 
support provided through this policy. 
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Strength of External 
Support and Implications 

What do we make of the patterns of 
support described in the previous 
section of this report? What would we 
predict to be the likely impact on 
probation schools? One obvious 
conclusion is that, through its allocation 
of substantial resources and attention to 
the external assistance efforts, CPS has 
demonstrated not only a recognition of 
the need for capacity building in its low- 
performing schools but a commitment 
to its realization. Few jurisdictions have, 
from the outset, designed an 
accountability system with this level of 
commitment to school assistance. 
Moreover, the incorporation of multiple 
layers of assistance reflects the district's 
recognition of the variability in school 
contexts, focus on the "fit" between the 
providers and the schools, and attempt 
to involve the broad educational 
community, including institutions of 
higher education, in improvement 
efforts. 

Unfortunately, our second main 
conclusion is that despite the millions of 
dollars put into assistance, the support 
component of the probation policy is 
simply too weak to make it through the 
many layers of implementation to 
significantly alter classroom instruction. 
This conclusion is supported by our case 
study data. A strong, direct, and 
observable impact of the external 
partners on classroom instruction was 
only evident in 2 of the 10 schools. In 
the other schools, respondents pointed 
to a number of problems in the 
provision of support: high external 



partner turnover, inadequate expertise 
of the consultants working with the 
school, inconsistency between the 
external partner program and school 
philosophy, redundancy of the external 
partner program with other programs, 
and lack of coimection between the 
external partner's professional 
development and the identified needs of 
the school. These problems, combined 
with the low intensity of assistance, 
limited the impact of the external 
partners on instruction. Moreover, the 
vague focus of the support providers 
became further diluted as it worked its 
way through the particularities of 
individual school contexts. In most of 
the schools in our study, we found little 
consistent, concrete guidance around 
literacy instruction and variable 
evidence of the impact of assistance on 
the behavior of professionals in the 
school. 

Policy Strength 

In this final section of this report, we 
detail our conclusions about the overall 
strength of the probation policy and 
suggest implications for change. We do 
so by re-exploring our findings in light 
of a framework developed by Andrew 
Porter and his colleagues to examine 
curriculum policy.^^ This framework 
analyzes variation in policy strength as 
occurring along four dimensions: 
prescriptiveness, consistency, authority. 



” These definitions differ slightly from Porter, 
Floden, Freeman, Schmidt, and Schwille (1988). 
For example, we include resources committed to 
a policy as part of the poiver behind the policy. 
We distinguish this from authority in that power 
implies concrete or material consequences or 
inducements. 
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and power (Porter, Floden, Freeman, 
Schmidt, & Schwille, 1988). 
Prescriptiveness refers to the degree of 
specificity in the policy. In curriculum 
policy, for example, broad standards 
would be less prescriptive than a more 
elaborated scope and sequence, which 
would be less prescriptive that a 
mandated curriculum and instructional 
materials. Consistency refers to the 
coherence among different policies and 
policy elements and is, of course, a 
central tenet of standards-based 
systemic reform. Policy authority may 
derive from varying sources, such as 
professional expertise or hierarchical 
position. And policies get their power 
from the extent of the resources 
committed to their enactment as well as 
from the constellation of rewards and 
sanctions accompanying them. 

It is important to note that these 
attributes of policy are not necessarily 
additive or even altogether independent 
of one another. For example, consistency 
among policies may have a 
multiplicative (rather than merely 
additive) effect on the strength of each. 
By contrast, a high degree of 
prescriptiveness may actually 
undermine a policy's authority by 
failing to honor the professional 
knowledge and discretion of teachers. 
Policy strength, therefore, may derive 
more from achieving a proper balance of 
the components outlined by Porter et al. 
(1988) rather than from high levels of 
each. Be that as it may, the bottom line is 
that for policy to have an impact on 
instruction, implementors need to know 
what policymakers want them to do, 
policy goals have to be consistent with 
other demands on practitioners' 



attention and time, and practitioners 
need to have a reason to expend effort in 
the direction of the policy. The authority 
and power behind the policy are meant 
to provide that reason. 

How does the external support 
system associated with probation fare 
along these dimensions? 

Prescriptiveness/Specificity 

We have already noted that the 
support policy is purposefully non- 
prescriptive in order to allow for a 
better match between partners and the 
needs of their schools. On the one hand, 
the market approach to partner selection 
engenders the desired variation and 
flexibility in partner strategies. On the 
other, the vagueness in the purposes of 
assistance may undermine quality 
control and overall effectiveness. For 
example, the requirements of the RFP 
(Request for Proposals) for prospective 
partners did not stipulate a focus on 
improving literacy instruction, so it 
should not be surprising that few 
partners had a coherent literacy 
approach. Yet, the research suggests 
that without such an approach, there is 
little hope of significantly altering the 
ways teachers teach reading. In 
addition, the huge scope of the five 
areas designated in the RFP — 
increasing student achievement, 
improving school leadership, 
establishing a student-centered learning 
environment, providing effective 
professional development 
opportunities, and promoting parent/ 
community partnerships — may 
encourage partners to spread their 
efforts too broadly, diluting their 
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potential for impact. Indeed, the data 
presented in the previous section 
suggest that partners were all over the 
map in their orientation to 
improvement, often overlapping or 
duplicating the work of other providers. 

Perhaps if probation schools had the 
capacity to carefully evaluate the 
partners' approaches and track records, 
such variation would not be a problem. 
However, most probation schools have 
difficulty identifying their own needs, 
much less the promises and designs of 
external providers. Our interviews with 
principals indicate that the building 
leaders in schools that came off 
probation quickly (within two years) 
were much more systematic in their 
selection of partners than were those 
principals of schools that remained on 
probation. The successful principals 
reported that they already had a vision 
for what they wanted to accomplish in 
their improvement efforts and chose a 
partner to support those efforts. In the 
less successful cases, partner selection 
seemed to have been a product of who 
the principal had met or worked with 
previously. 

Probation managers did not fare 
much better than external partners in 
terms of the specificity of their role and 
focus. Probation managers reported that 
they received very little guidance as to 
what they should focus on in the schools 
and no training on how to be effective 
interveners and mentors. Rather, this 
part of the program seemed to rest on 
the faith that probation managers would 
choose the right targets based on their 
own prior success. Unsure of their 
actual role, however, probation 



managers were forced to invent it 
themselves and hope for the best. 

Finally, it is important to note that 
the one area of prescriptiveness in the 
policy was its emphasis on ITBS scores, 
which may have had the most 
constraining influence on practice by 
encouraging low-level test preparation 
activities. 

Consistency 

On the surface, the support system is 
highly consistent with other policies, 
particularly in the emphasis on raising 
test scores. The fact that the ITBS is used 
for student promotion decisions as well 
as for probation reinforces the test's 
impact (for better or worse) on both 
policies. By contrast, both the state test 
and the Chicago Academic Standards 
appear to take a very secondary role in 
the work of the providers and the 
schools. Claims of alignment of the tests 
and the standards are commonplace but 
the evidence is slim. Moreover, it is the 
ITBS and not the standards that drives 
the curriculum. Partners talked about 
ensuring coverage of tested skills and 
mapping curriculum onto the test 
specifications (rather than onto 
standards themselves). To the extent 
that the Chicago Academic Standards 
are supposed to be at the core of the 
reform efforts and that the ITBS is an 
inadequate assessment of those 
standards, this emphasis on the test in 
curriculum development is problematic. 

Inconsistencies among providers are 
also a substantial design and 
implementation issue in the current 
support system. Not only did the targets 
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of external partners and probation 
managers overlap in the schools, often 
with inadequate communication 
between these groups, but also the 
addition of the regional education office 
and Office of Accountability facilitators 
extended the likelihood of conflicting 
messages to schools. Similarly, the 
existence of multiple partners not 
coimected with probation enhanced the 
likelihood of fragmentation and 
inadequate focus. 

Finally, as the district seeks to 
tighten its control through greater 
specificity, new avenues of potential 
inconsistency open up due to variation 
previously created by the market. For 
example, the Targeted Assistance 
Program, which provides additional 
(federal) monies to low-performing 
schools to reduce class size in the 
primary grades, required that 
participating schools institute 90-minute 
literacy blocks in the morning and that 
reading instruction during these blocks 
be at grade level for all students. This 
requirement directly contradicted 
literacy programs already implemented 
by some probation schools that grouped 
students by instructional level for 
reading. Success for All is the most 
obvious example of such a program. 
This inconsistency forces schools to 
refuse the additional resources or 
undermine what may be a functional 
reading program. 

Authority 

At first blush, the assistance 
components of the probation policy 
carry considerable administrative 
authority. Assistance is not voluntary as 



it is in some jurisdictions (e.g., 
Kentucky). Schools must contract with a 
partner and must have a probation 
manager. Moreover, the manager is at 
the school in part to monitor the 
improvement efforts, a role that 
suggests additional authority. 

Once the providers are inside the 
school, however, their authority 
becomes ambiguous at best. For 
example, the ability of schools to "fire" 
an external partner makes the partner 
vulnerable, transforming the role from 
one of a partner to that of a vendor. 
Vendors sell services, which consumers 
are under no obligation to accept or 
utilize. In addition, while probation 
managers have responsibility for 
monitoring the implementation of the 
School Improvement Plan for 
Advancing Academic Achievement, 
they have no clear authority to make 
decisions in the school. In fact, district 
administrators repeatedly commented 
that the probation managers were not 
co-principals and did not have decision- 
making authority, nor did they appear 
to have the ability to sanction either 
principals or other persoimel. 

Comments about their lack of authority 
and the confusion in their role were 
commonplace in our interviews with 
probation managers. 

While support providers may not 
have had clear lines of administrative 
authority in the schools, they did, of 
course, have access to other kinds of 
authority — primarily that which comes 
with professional expertise. To the 
extent that partners were able to 
convince their schools — including the 
teachers — that they brought a needed 
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form of expertise, they were able to gain 
professional authority. The policy's 
reliance on the professional authority of 
support providers increases the 
importance of selection — both at the 
district level, where partners are 
approved, and at the school level, where 
they are actually chosen. 

Evidence of effectiveness, on which 
basis selection should be made, is 
scarce, however. Moreover, as stated 
above, a school's choice of partner 
appeared to be made largely on the 
basis of who was known to schools 
rather than through a careful 
consideration of the approach or 
philosophy of the partner. Furthermore, 
to be selected, external partners may 
have over-promised what they could 
offer. Failure to follow through on these 
promises then decreased the school's 
trust in the partner's ability to do what 
it claimed and, thus, decreased the 
partner's authority. A more serious 
threat to professional authority, 
however, derived from the quality of 
professional development. Complaints 
about the usefulness of the staff 
development provided through the 
external partners were commonplace in 
our school-level interviews. 

Probation managers also faced the 
need to establish professional authority 
if they were to gain the ear of the 
principal. Those who had successes in 
schools that were somewhat similar to 
their assigned probation schools had 
better standing in this regard than those 
whose contexts were very different from 
that faced by their principal mentee. 



Power 

Power consists both of the resources 
committed to the policy and of the 
structure of incentives that surround the 
policy. Again, we find several indicators 
of the power behind probation, in 
general, and the provision of support, in 
particular. With respect to incentives, 
the fact that probation schools, in 
theory, face more severe consequences 
should their aggregate achievement not 
improve would seem to provide a 
powerful incentive for them to attend to 
and put to use the assistance they 
receive from whatever source. However, 
our case study data — and indeed the 
interviews of district officials — 
indicated that few persoimel in these 
schools were motivated by the threat of 
reconstitution or other sanctions 
(Beimett, 2001; Fiimigan & Gross, 2001). 
Few believed that the school would be 
closed down or reconstituted; indeed 
district officials declared early on that 
they would not reconstitute elementary 
schools. We should note that this 
situation may be changing as some 
schools linger on probation indefinitely. 

For the threat of sanctions to have an 
impact on assistance, school persoimel 
must see a direct link between that 
assistance and getting off probation. 

This brings us back to the strength of the 
professional authority both partners and 
probation managers command. To the 



Here the line between authority and power 
becomes somewhat blurred. We mentioned in 
the previous section that probation managers 
lacked not only general decision-making 
authority, but also the ability (or power) to 
sanction principals or school staff. 
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extent that this authority was weak or 
that teachers simply did not make the 
link (or believed that even with 
assistance their school would not be able 
to succeed), other incentives to 
participate in professional development 
would have been particularly valuable. 
We saw little evidence of such 
additional incentives.^^ 

Perhaps the most important aspect 
of power in this policy, however, is 
simply the very substantial resources 
put into it. Yet, despite this large 
financial commitment, the assistance 
provided was of very limited intensity. 

If Supovitz and Turner (2000) are 
correct, it takes from 80 to 160 hours of 
professional development in a content 
area to see significant changes in 
practice. However, external partners 
spent approximately two days per week 
in a school and worked with individual 
teachers only a few times a year. In 
addition, workshops were generally 
half-day to full-day affairs. In some 
cases, those who attended the 
workshops were charged with 
disseminating the knowledge to others 
in the school; yet by all accounts, 
workshop attendees had a very limited 
understanding of the content 
themselves. This made it difficult for 
them to disseminate it even under the 
best of circumstances — but often the 
school provided little structural support 
for that dissemination either. External 
partner consultants may have tried to 



** One exception was the offering of school- 
based professional developmenf courses for 
college credif. The facf fhaf feachers would be 
awarded credif increased fheir mofivafion fo 
parficipafe. 



assist, but were generally in the schools 
too little time themselves. Furthermore, 
the workshops the external partners 
provided were usually uncoimected, did 
not link to classroom interaction, and 
did not involve the depth that the 
literature on professional development 
suggests is necessary for professional 
learning. 

Overall, we tend to agree with prior 
researchers who argue that external 
providers simply caimot provide the 
intensity of professional development 
needed to change classroom practice 
within the current resource constraints. 
We did not see much indication of 
external partners pushing teachers or 
administrators to participate in deeper 
forms of professional development to 
supplement their learning. Success for 
All and the Center for School 
Improvement may be exceptions to this 
pattern. Both sponsored intensive and 
extensive professional development in 
literacy for the individuals who then 
became the literacy coordinators for 
their schools and who provided on-the- 
ground curriculum development, 
professional learning, and follow-up in 
the area of literacy. But these were the 
exceptions, and even in these cases, the 
training for most teachers in the schools 
was quite limited. 

Our main conclusion, then, is that 
the support provided to schools is too 
fragmentary and weak to have the deep 
effect needed to change instruction, 
though it may improve various aspects 
of the instructional environment. The 
observed weaknesses appear 
attributable, in part, to implementation 
issues and, in part, to design. 
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Implications 

Several implications for 
strengthening CPS's support policy 
emerge from this analysis. Here we 
suggest seven: 

1. Constrain the market choice, both 
with respect to direction and in 
terms of quality. We believe the 
provision of assistance would be 
improved if the number of partners 
were limited to a handful of proven 
groups that were required to offer or 
support a coherent and 
comprehensive instructional 
program at least in the area of 
literacy. In this more constrained 
market approach, the district could 
better ensure both quality and 
appropriate variation and flexibility, 
assisting low-capacity schools to 
make more useful choices. 

2. Clarify roles of support providers. 

The existence of multiple partners 
and providers almost guarantees 
fragmentation at the school level. We 
suggest that the roles of probation 
managers, regional education office 
staff, and Office of Accountability 
facilitators be consolidated into a 
single and clear line of authority in 
the schools and that this authority 
work closely with the external 
partner to help the school develop 
and implement a coherent 
instructional program. 

3. Develop opportunities for 
learning/sharing among partners 
and probation managers. We 

strongly urge the district to design 
professional development for these 



support providers and opportunities 
for them to reflect collectively on 
their work, its effects, and resulting 
lessons. 

4. Discourage schools from 
developing multiple and 
fragmented partnerships by giving 
the external partner the authority to 
coordinate assistance from all 
external sources while the school is 
on probation. 

5. Stimulate the development of in- 
depth, content-based professional 
development. An example of such 
staff development might be the two- 
week session on reading instruction 
for Targeted Assistance Program 
teachers held in the fall of 2000. 
Despite the implementation 
problems of short notice and 
inconsistency with some literacy 
programs, this effort to foster deeper 
content and pedagogical content 
knowledge in literacy was an 
important step and should be 
pursued further. 

6. Connect assistance to the standards, 
not just the test. We observed a real 
problem of teaching to the test in 
several of our case study schools, a 
problem that is exacerbated by the 
lack of attention to the standards and 
the reliance on a single indicator for 
both student promotion and school 
probation. As long as the ITBS is the 
sole indicator and everything rides 
on it, it will become the de facto 
standards of the district. 
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7. Promote better use of data in schools 
by fostering the development and 
implementation of multiple 
diagnostic tools such as ruiming 
records, periodic curriculum-based 
assessments, and examination of 
more extensive examples of student 
work. 

Conclusion 

The emphasis on providing low- 
performing schools regular, long-term 
assistance to improve is a hallmark of 
the CPS school accountability system, 
one that we applaud as an example to 
other jurisdictions. However, our data 
suggest that in the lowest performing 
schools, current assistance efforts are 
simply not strong enough to overcome 
the deep problems in educator and 
organizational capacity necessary to 
fundamentally improve instruction. We 
encourage the district to revisit the 
design of the system to address the 
problems outlined in this report. While 
many of our recommendations do not 
require additional resources, others 
clearly do. In particular, the intensity of 
professional development opportunities 
afforded to teachers must be enhanced if 
the policy is to have its desired effect. 

As Cohen and Ball (1999) point out, 
teachers are the mediators of all 
instructional interventions. Their 
capacity and motivation are critical. 
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Appendix A. Description of External Partners Participating in 
Study 


Name of Organization 


Type of Support 


Number of 
Schools 
Served 
During Study 


Accountabiiity Direct instruction Model 


Instruction— Direct Instruction K-6 


4 


Achieving High Standards Project, 
University of liiinois-Chicago 


Instruction— Four Blocks of Literacy 


3 


Brad Frieswyk Group 


Instruction— Direct Instruction K-8 


6* 


Center for Schooi Improvement, 
University of Chicago 


Instruction— Literacy, STEP Assessments 
Organization— Academic and Social 
Supports System, Principal Network, 
Grand Rounds Review 


2 


Center for Urban Education, DePaui University 


Instruction and Organization 


33 


Community for Learning, Tempie University 


Instruction— Adaptive Learning Environment 
Organization— Degree of Implementation 
Report 


10 


interactive Teaching and Learning, 
Northeastern liiinois University 


Instruction 

Organization— Needs Assessment, 
Planning Assistance 


8 


Nationai Schooi Services/I Had a Dream, Inc. 


Instruction 


5 


School Achievement Structure, 
DePaul University 


Organization— Ten Structured Routines, 
Self-study 


12 


Small Schools Workshop, 
University of Illinois-Chicago 


Organization— Small Schools within Schools 


3 


Valina Miller Group 


Instruction and Organization 


3 



This partner was subcontracted by School Achievement Structure to work in four probation schools. 



